codec: mirror benchmarks into bench sub-directory

README.md

@@ -5,136 +5,173 @@
 [![rcard](https://goreportcard.com/badge/github.com/ugorji/go/codec?v=4)](https://goreportcard.com/report/github.com/ugorji/go/codec)
 [![License](http://img.shields.io/badge/license-mit-blue.svg?style=flat-square)](https://raw.githubusercontent.com/ugorji/go/master/LICENSE)
 
-# Codec aka go-codec
+# go-codec
 
-This repository contains the `go-codec` library,
-a High Performance and Feature-Rich Idiomatic encode/decode and rpc library for:
-
-  - msgpack: https://github.com/msgpack/msgpack
-  - binc:    http://github.com/ugorji/binc
-  - cbor:    http://cbor.io http://tools.ietf.org/html/rfc7049
-  - json:    http://json.org http://tools.ietf.org/html/rfc7159
-  - simple: 
+This repository contains the `go-codec` library.
 
 To install:
 
-    go get github.com/ugorji/go/codec
+```
+go get github.com/ugorji/go/codec
+```
+
+# Package Documentation
+
+
+Package codec provides a High Performance, Feature-Rich Idiomatic Go 1.4+
+codec/encoding library for binc, msgpack, cbor, json.
 
-*Note: the import path `github.com/ugorji/go/codec` is maintained for backwards compatibility,
-and enforced using the files: `go.mod` and `0_importpath.go`.* 
+Supported Serialization formats are:
 
-This package will carefully use 'unsafe' for performance reasons in specific places.
-You can build without unsafe use by passing the safe or appengine tag
-i.e. 'go install -tags=safe ...'. Note that unsafe is only supported for the last 3
-go sdk versions e.g. current go release is go 1.9, so we support unsafe use only from
-go 1.7+ . This is because supporting unsafe requires knowledge of implementation details.
+```
+    - msgpack: https://github.com/msgpack/msgpack
+    - binc:    http://github.com/ugorji/binc
+    - cbor:    http://cbor.io http://tools.ietf.org/html/rfc7049
+    - json:    http://json.org http://tools.ietf.org/html/rfc7159
+    - simple:
+```
 
-Online documentation: http://godoc.org/github.com/ugorji/go/codec  
-Detailed Usage/How-to Primer: http://ugorji.net/blog/go-codec-primer
+This package will carefully use 'unsafe' for performance reasons in specific
+places. You can build without unsafe use by passing the safe or appengine
+tag i.e. 'go install -tags=safe ...'. Note that unsafe is only supported for
+the last 3 go sdk versions e.g. current go release is go 1.9, so we support
+unsafe use only from go 1.7+ . This is because supporting unsafe requires
+knowledge of implementation details.
 
-The idiomatic Go support is as seen in other encoding packages in
-the standard library (ie json, xml, gob, etc).
+For detailed usage information, read the primer at
+http://ugorji.net/blog/go-codec-primer .
+
+The idiomatic Go support is as seen in other encoding packages in the
+standard library (ie json, xml, gob, etc).
 
 Rich Feature Set includes:
 
-  - Simple but extremely powerful and feature-rich API
-  - Support for go1.4 and above, while selectively using newer APIs for later releases
-  - Excellent code coverage ( > 90% )
-  - Very High Performance.
-    Our extensive benchmarks show us outperforming Gob, Json, Bson, etc by 2-4X.
-  - Careful selected use of 'unsafe' for targeted performance gains.
-    100% mode exists where 'unsafe' is not used at all.
-  - Lock-free (sans mutex) concurrency for scaling to 100's of cores
-  - In-place updates during decode, with option to zero the value in maps and slices prior to decode
-  - Coerce types where appropriate
-    e.g. decode an int in the stream into a float, decode numbers from formatted strings, etc
-  - Corner Cases: 
-    Overflows, nil maps/slices, nil values in streams are handled correctly
-  - Standard field renaming via tags
-  - Support for omitting empty fields during an encoding
-  - Encoding from any value and decoding into pointer to any value
-    (struct, slice, map, primitives, pointers, interface{}, etc)
-  - Extensions to support efficient encoding/decoding of any named types
-  - Support encoding.(Binary|Text)(M|Unm)arshaler interfaces
-  - Support IsZero() bool to determine if a value is a zero value.
-    Analogous to time.Time.IsZero() bool.
-  - Decoding without a schema (into a interface{}).
-    Includes Options to configure what specific map or slice type to use
-    when decoding an encoded list or map into a nil interface{}
-  - Mapping a non-interface type to an interface, so we can decode appropriately
-    into any interface type with a correctly configured non-interface value.
-  - Encode a struct as an array, and decode struct from an array in the data stream
-  - Option to encode struct keys as numbers (instead of strings)
-    (to support structured streams with fields encoded as numeric codes)
-  - Comprehensive support for anonymous fields
-  - Fast (no-reflection) encoding/decoding of common maps and slices
-  - Code-generation for faster performance.
-  - Support binary (e.g. messagepack, cbor) and text (e.g. json) formats
-  - Support indefinite-length formats to enable true streaming 
-    (for formats which support it e.g. json, cbor)
-  - Support canonical encoding, where a value is ALWAYS encoded as same sequence of bytes.
-    This mostly applies to maps, where iteration order is non-deterministic.
-  - NIL in data stream decoded as zero value
-  - Never silently skip data when decoding.
-    User decides whether to return an error or silently skip data when keys or indexes
-    in the data stream do not map to fields in the struct.
-  - Encode/Decode from/to chan types (for iterative streaming support)
-  - Drop-in replacement for encoding/json. `json:` key in struct tag supported.
-  - Provides a RPC Server and Client Codec for net/rpc communication protocol.
-  - Handle unique idiosyncrasies of codecs e.g. 
-    - For messagepack, configure how ambiguities in handling raw bytes are resolved 
-    - For messagepack, provide rpc server/client codec to support
-      msgpack-rpc protocol defined at:
-      https://github.com/msgpack-rpc/msgpack-rpc/blob/master/spec.md
+```
+    - Simple but extremely powerful and feature-rich API
+    - Support for go1.4 and above, while selectively using newer APIs for later releases
+    - Excellent code coverage ( > 90% )
+    - Very High Performance.
+      Our extensive benchmarks show us outperforming Gob, Json, Bson, etc by 2-4X.
+    - Careful selected use of 'unsafe' for targeted performance gains.
+      100% mode exists where 'unsafe' is not used at all.
+    - Lock-free (sans mutex) concurrency for scaling to 100's of cores
+    - In-place updates during decode, with option to zero value in maps and slices prior to decode
+    - Coerce types where appropriate
+      e.g. decode an int in the stream into a float, decode numbers from formatted strings, etc
+    - Corner Cases:
+      Overflows, nil maps/slices, nil values in streams are handled correctly
+    - Standard field renaming via tags
+    - Support for omitting empty fields during an encoding
+    - Encoding from any value and decoding into pointer to any value
+      (struct, slice, map, primitives, pointers, interface{}, etc)
+    - Extensions to support efficient encoding/decoding of any named types
+    - Support encoding.(Binary|Text)(M|Unm)arshaler interfaces
+    - Support IsZero() bool to determine if a value is a zero value.
+      Analogous to time.Time.IsZero() bool.
+    - Decoding without a schema (into a interface{}).
+      Includes Options to configure what specific map or slice type to use
+      when decoding an encoded list or map into a nil interface{}
+    - Mapping a non-interface type to an interface, so we can decode appropriately
+      into any interface type with a correctly configured non-interface value.
+    - Encode a struct as an array, and decode struct from an array in the data stream
+    - Option to encode struct keys as numbers (instead of strings)
+      (to support structured streams with fields encoded as numeric codes)
+    - Comprehensive support for anonymous fields
+    - Fast (no-reflection) encoding/decoding of common maps and slices
+    - Code-generation for faster performance.
+    - Support binary (e.g. messagepack, cbor) and text (e.g. json) formats
+    - Support indefinite-length formats to enable true streaming
+      (for formats which support it e.g. json, cbor)
+    - Support canonical encoding, where a value is ALWAYS encoded as same sequence of bytes.
+      This mostly applies to maps, where iteration order is non-deterministic.
+    - NIL in data stream decoded as zero value
+    - Never silently skip data when decoding.
+      User decides whether to return an error or silently skip data when keys or indexes
+      in the data stream do not map to fields in the struct.
+    - Detect and error when encoding a cyclic reference (instead of stack overflow shutdown)
+    - Encode/Decode from/to chan types (for iterative streaming support)
+    - Drop-in replacement for encoding/json. `json:` key in struct tag supported.
+    - Provides a RPC Server and Client Codec for net/rpc communication protocol.
+    - Handle unique idiosyncrasies of codecs e.g.
+      - For messagepack, configure how ambiguities in handling raw bytes are resolved
+      - For messagepack, provide rpc server/client codec to support
+        msgpack-rpc protocol defined at:
+        https://github.com/msgpack-rpc/msgpack-rpc/blob/master/spec.md
+```
+
 
 ## Extension Support
 
-Users can register a function to handle the encoding or decoding of
-their custom types.
+Users can register a function to handle the encoding or decoding of their
+custom types.
 
 There are no restrictions on what the custom type can be. Some examples:
 
+```go
     type BisSet   []int
     type BitSet64 uint64
     type UUID     string
     type MyStructWithUnexportedFields struct { a int; b bool; c []int; }
     type GifImage struct { ... }
+```
+
+As an illustration, MyStructWithUnexportedFields would normally be encoded
+as an empty map because it has no exported fields, while UUID would be
+encoded as a string. However, with extension support, you can encode any of
+these however you like.
 
-As an illustration, MyStructWithUnexportedFields would normally be
-encoded as an empty map because it has no exported fields, while UUID
-would be encoded as a string. However, with extension support, you can
-encode any of these however you like.
 
 ## Custom Encoding and Decoding
 
-This package maintains symmetry in the encoding and decoding halfs.
-We determine how to encode or decode by walking this decision tree
+This package maintains symmetry in the encoding and decoding halfs. We
+determine how to encode or decode by walking this decision tree
 
-  - is type a codec.Selfer?
-  - is there an extension registered for the type?
-  - is format binary, and is type a encoding.BinaryMarshaler and BinaryUnmarshaler?
-  - is format specifically json, and is type a encoding/json.Marshaler and Unmarshaler?
-  - is format text-based, and type an encoding.TextMarshaler and TextUnmarshaler?
-  - else we use a pair of functions based on the "kind" of the type e.g. map, slice, int64, etc
+```
+    - is type a codec.Selfer?
+    - is there an extension registered for the type?
+    - is format binary, and is type a encoding.BinaryMarshaler and BinaryUnmarshaler?
+    - is format specifically json, and is type a encoding/json.Marshaler and Unmarshaler?
+    - is format text-based, and type an encoding.TextMarshaler and TextUnmarshaler?
+    - else we use a pair of functions based on the "kind" of the type e.g. map, slice, int64, etc
+```
 
 This symmetry is important to reduce chances of issues happening because the
 encoding and decoding sides are out of sync e.g. decoded via very specific
 encoding.TextUnmarshaler but encoded via kind-specific generalized mode.
 
-Consequently, if a type only defines one-half of the symmetry
-(e.g. it implements UnmarshalJSON() but not MarshalJSON() ),
-then that type doesn't satisfy the check and we will continue walking down the
-decision tree.
+Consequently, if a type only defines one-half of the symmetry (e.g. it
+implements UnmarshalJSON() but not MarshalJSON() ), then that type doesn't
+satisfy the check and we will continue walking down the decision tree.
+
 
 ## RPC
 
-RPC Client and Server Codecs are implemented, so the codecs can be used
-with the standard net/rpc package.
+RPC Client and Server Codecs are implemented, so the codecs can be used with
+the standard net/rpc package.
+
 
 ## Usage
 
-Typical usage model:
+The Handle is SAFE for concurrent READ, but NOT SAFE for concurrent
+modification.
+
+The Encoder and Decoder are NOT safe for concurrent use.
+
+Consequently, the usage model is basically:
+
+```
+    - Create and initialize the Handle before any use.
+      Once created, DO NOT modify it.
+    - Multiple Encoders or Decoders can now use the Handle concurrently.
+      They only read information off the Handle (never write).
+    - However, each Encoder or Decoder MUST not be used concurrently
+    - To re-use an Encoder/Decoder, call Reset(...) on it first.
+      This allows you use state maintained on the Encoder/Decoder.
+```
 
+Sample usage model:
+
+```go
     // create and configure Handle
     var (
       bh codec.BincHandle
@@ -179,37 +216,89 @@ Typical usage model:
     rpcCodec := codec.GoRpc.ClientCodec(conn, h)
     //OR rpcCodec := codec.MsgpackSpecRpc.ClientCodec(conn, h)
     client := rpc.NewClientWithCodec(rpcCodec)
+```
+
 
 ## Running Tests
 
 To run tests, use the following:
 
+```
     go test
+```
 
 To run the full suite of tests, use the following:
 
+```
     go test -tags alltests -run Suite
+```
 
 You can run the tag 'safe' to run tests or build in safe mode. e.g.
 
+```
     go test -tags safe -run Json
     go test -tags "alltests safe" -run Suite
+```
+
 
 ## Running Benchmarks
 
 Please see http://github.com/ugorji/go-codec-bench .
 
+
 ## Caveats
 
-Struct fields matching the following are ignored during encoding and decoding
+Struct fields matching the following are ignored during encoding and
+decoding
 
-  - struct tag value set to -
-  - func, complex numbers, unsafe pointers
-  - unexported and not embedded
-  - unexported and embedded and not struct kind
-  - unexported and embedded pointers (from go1.10)
+```
+    - struct tag value set to -
+    - func, complex numbers, unsafe pointers
+    - unexported and not embedded
+    - unexported and embedded and not struct kind
+    - unexported and embedded pointers (from go1.10)
+```
 
 Every other field in a struct will be encoded/decoded.
 
-Embedded fields are encoded as if they exist in the top-level struct,
-with some caveats. See Encode documentation.
+Embedded fields are encoded as if they exist in the top-level struct, with
+some caveats. See Encode documentation.
+
+## Exported Package API
+
+```go
+const CborStreamBytes byte = 0x5f ...
+const GenVersion = 10
+var GoRpc goRpc
+var MsgpackSpecRpc msgpackSpecRpc
+func GenHelperDecoder(d *Decoder) (gd genHelperDecoder, dd genHelperDecDriver)
+func GenHelperEncoder(e *Encoder) (ge genHelperEncoder, ee genHelperEncDriver)
+type BasicHandle struct{ ... }
+type BincHandle struct{ ... }
+type BytesExt interface{ ... }
+type CborHandle struct{ ... }
+type DecodeOptions struct{ ... }
+type Decoder struct{ ... }
+    func NewDecoder(r io.Reader, h Handle) *Decoder
+    func NewDecoderBytes(in []byte, h Handle) *Decoder
+type EncodeOptions struct{ ... }
+type Encoder struct{ ... }
+    func NewEncoder(w io.Writer, h Handle) *Encoder
+    func NewEncoderBytes(out *[]byte, h Handle) *Encoder
+type Ext interface{ ... }
+type Handle interface{ ... }
+type InterfaceExt interface{ ... }
+type JsonHandle struct{ ... }
+type MapBySlice interface{ ... }
+type MissingFielder interface{ ... }
+type MsgpackHandle struct{ ... }
+type MsgpackSpecRpcMultiArgs []interface{}
+type RPCOptions struct{ ... }
+type Raw []byte
+type RawExt struct{ ... }
+type Rpc interface{ ... }
+type Selfer interface{ ... }
+type SimpleHandle struct{ ... }
+type TypeInfos struct{ ... }
+    func NewTypeInfos(tags []string) *TypeInfos
+```

codec/0_doc.go

@@ -14,10 +14,6 @@ Supported Serialization formats are:
   - json:    http://json.org http://tools.ietf.org/html/rfc7159
   - simple:
 
-To install:
-
-    go get github.com/ugorji/go/codec
-
 This package will carefully use 'unsafe' for performance reasons in specific places.
 You can build without unsafe use by passing the safe or appengine tag
 i.e. 'go install -tags=safe ...'. Note that unsafe is only supported for the last 3

codec/README.md

@@ -1 +0,0 @@
-../README.md

codec/bench/README.md

@@ -0,0 +1,153 @@
+# go-codec-bench
+
+This is a comparison of different binary and text encodings.
+
+We compare the codecs provided by github.com/ugorji/go/codec package,
+against other libraries:
+
+[github.com/ugorji/go/codec](http://github.com/ugorji/go) provides:
+
+  - msgpack: [http://github.com/msgpack/msgpack] 
+  - binc:    [http://github.com/ugorji/binc]
+  - cbor:    [http://cbor.io] [http://tools.ietf.org/html/rfc7049]
+  - simple: 
+  - json:    [http://json.org] [http://tools.ietf.org/html/rfc7159] 
+
+Other codecs compared include:
+
+  - [gopkg.in/vmihailenco/msgpack.v2](http://gopkg.in/vmihailenco/msgpack.v2)
+  - [gopkg.in/mgo.v2/bson](http://gopkg.in/mgo.v2/bson)
+  - [github.com/davecgh/go-xdr/xdr2](https://godoc.org/github.com/davecgh/go-xdr/xdr)
+  - [github.com/Sereal/Sereal/Go/sereal](https://godoc.org/github.com/Sereal/Sereal/Go/sereal)
+  - [code.google.com/p/cbor/go](http://code.google.com/p/cbor/go)
+  - [github.com/tinylib/msgp](http://github.com/tinylib/msgp)
+  - [github.com/tinylib/msgp](http://godoc.org/github.com/tinylib/msgp)
+  - [github.com/pquerna/ffjson/ffjson](http://godoc.org/github.com/pquerna/ffjson/ffjson)
+  - [bitbucket.org/bodhisnarkva/cbor/go](http://godoc.org/bitbucket.org/bodhisnarkva/cbor/go)
+  - [github.com/json-iterator/go](http://godoc.org/github.com/json-iterator/go)
+  - [github.com/mailru/easyjson](http://godoc.org/github.com/mailru/easyjson)
+  
+# Data
+
+The data being serialized is a `TestStruc` randomly generated values.
+See https://github.com/ugorji/go-codec-bench/blob/master/codec/values_test.go for the
+definition of the TestStruc.
+
+# Run Benchmarks
+
+See  https://github.com/ugorji/go-codec-bench/blob/master/codec/bench.sh 
+for how to download the external libraries which we benchmark against,
+generate the files for the types when needed, 
+and run the suite of tests.
+
+The 3 suite of benchmarks are
+
+  - CodecSuite
+  - XSuite
+  - CodecXSuite
+
+```
+# Note that `bench.sh` may be in the codec sub-directory, and should be run from there.
+
+# download the code and all its dependencies
+./bench.sh -d
+
+# code-generate files needed for benchmarks against ffjson, easyjson, msgp, etc
+./bench.sh -c
+
+# run the full suite of tests
+./bench.sh -s
+
+# Below, see how to just run some specific suite of tests, knowing the right tags and flags ...
+# See bench.sh for different iterations
+
+# Run suite of tests in default mode (selectively using unsafe in specific areas)
+go test -tags "alltests x" -bench "CodecXSuite" -benchmem 
+# Run suite of tests in safe mode (no usage of unsafe)
+go test -tags "alltests x safe" -bench "CodecXSuite" -benchmem 
+# Run suite of tests in codecgen mode, including all tests which are generated (msgp, ffjson, etc)
+go test -tags "alltests x generated" -bench "CodecXGenSuite" -benchmem 
+
+```
+
+# Issues
+
+The following issues are seen currently (11/20/2014):
+
+- _code.google.com/p/cbor/go_ fails on encoding and decoding the test struct
+- _github.com/davecgh/go-xdr/xdr2_ fails on encoding and decoding the test struct
+- _github.com/Sereal/Sereal/Go/sereal_ fails on decoding the serialized test struct
+
+# Representative Benchmark Results
+
+Please see the [benchmarking blog post for detailed representative results](http://ugorji.net/blog/benchmarking-serialization-in-go).
+
+A snapshot of some results on my 2016 MacBook Pro is below.  
+**Note: errors are truncated, and lines re-arranged, for readability**.
+
+Below are results of running the entire suite on 2017-11-20 (ie running ./bench.sh -s).
+
+What you should notice:
+
+- Results get better with codecgen, showing about 20-50% performance improvement.
+  Users should carefully weigh the performance improvements against the 
+  usability and binary-size increases, as performance is already extremely good 
+  without the codecgen path.
+  
+See  https://github.com/ugorji/go-codec-bench/blob/master/bench.out.txt for latest run of bench.sh as of 2017-11-20
+
+* snippet of bench.out.txt, running without codecgen *
+```
+BenchmarkCodecXSuite/options-false.../Benchmark__Msgpack____Encode-8         	   10000	    183961 ns/op	   10224 B/op	      75 allocs/op
+BenchmarkCodecXSuite/options-false.../Benchmark__Binc_______Encode-8         	   10000	    206362 ns/op	   12551 B/op	      80 allocs/op
+BenchmarkCodecXSuite/options-false.../Benchmark__Simple_____Encode-8         	   10000	    193966 ns/op	   10224 B/op	      75 allocs/op
+BenchmarkCodecXSuite/options-false.../Benchmark__Cbor_______Encode-8         	   10000	    192666 ns/op	   10224 B/op	      75 allocs/op
+BenchmarkCodecXSuite/options-false.../Benchmark__Json_______Encode-8         	    3000	    475767 ns/op	   10352 B/op	      75 allocs/op
+BenchmarkCodecXSuite/options-false.../Benchmark__Std_Json___Encode-8         	    3000	    525223 ns/op	  256049 B/op	     835 allocs/op
+BenchmarkCodecXSuite/options-false.../Benchmark__Gob________Encode-8         	    5000	    270550 ns/op	  333548 B/op	     959 allocs/op
+BenchmarkCodecXSuite/options-false.../Benchmark__JsonIter___Encode-8         	    3000	    478130 ns/op	  183552 B/op	    3262 allocs/op
+BenchmarkCodecXSuite/options-false.../Benchmark__Bson_______Encode-8         	    2000	    747360 ns/op	  715539 B/op	    5629 allocs/op
+BenchmarkCodecXSuite/options-false.../Benchmark__VMsgpack___Encode-8         	    2000	    637388 ns/op	  320385 B/op	     542 allocs/op
+BenchmarkCodecXSuite/options-false.../Benchmark__Sereal_____Encode-8         	    5000	    361369 ns/op	  294541 B/op	    4286 allocs/op
+-------------------------------
+BenchmarkCodecXSuite/options-false.../Benchmark__Msgpack____Decode-8         	    5000	    370340 ns/op	  120352 B/op	    1210 allocs/op
+BenchmarkCodecXSuite/options-false.../Benchmark__Binc_______Decode-8         	    3000	    443650 ns/op	  126144 B/op	    1263 allocs/op
+BenchmarkCodecXSuite/options-false.../Benchmark__Simple_____Decode-8         	    3000	    381155 ns/op	  120352 B/op	    1210 allocs/op
+BenchmarkCodecXSuite/options-false.../Benchmark__Cbor_______Decode-8         	    5000	    370754 ns/op	  120352 B/op	    1210 allocs/op
+BenchmarkCodecXSuite/options-false.../Benchmark__Json_______Decode-8         	    2000	    719658 ns/op	  159289 B/op	    1478 allocs/op
+BenchmarkCodecXSuite/options-false.../Benchmark__Std_Json___Decode-8         	    1000	   2204258 ns/op	  276336 B/op	    6959 allocs/op
+BenchmarkCodecXSuite/options-false.../Benchmark__Gob________Decode-8         	    5000	    383884 ns/op	  256684 B/op	    3261 allocs/op
+BenchmarkCodecXSuite/options-false.../Benchmark__JsonIter___Decode-8         	    2000	    907079 ns/op	  301520 B/op	    7769 allocs/op
+BenchmarkCodecXSuite/options-false.../Benchmark__Bson_______Decode-8         	    2000	   1146851 ns/op	  373121 B/op	   15703 allocs/op
+```
+
+* snippet of bench.out.txt, running with codecgen *
+```
+BenchmarkCodecXGenSuite/options-false.../Benchmark__Msgpack____Encode-8         	   10000	    124729 ns/op	    6224 B/op	       7 allocs/op
+BenchmarkCodecXGenSuite/options-false.../Benchmark__Binc_______Encode-8         	   10000	    119745 ns/op	    6256 B/op	       7 allocs/op
+BenchmarkCodecXGenSuite/options-false.../Benchmark__Simple_____Encode-8         	   10000	    132501 ns/op	    6224 B/op	       7 allocs/op
+BenchmarkCodecXGenSuite/options-false.../Benchmark__Cbor_______Encode-8         	   10000	    129706 ns/op	    6224 B/op	       7 allocs/op
+BenchmarkCodecXGenSuite/options-false.../Benchmark__Json_______Encode-8         	    3000	    436958 ns/op	    6352 B/op	       7 allocs/op
+BenchmarkCodecXGenSuite/options-false.../Benchmark__Std_Json___Encode-8         	    3000	    539884 ns/op	  256049 B/op	     835 allocs/op
+BenchmarkCodecXGenSuite/options-false.../Benchmark__Gob________Encode-8         	    5000	    270663 ns/op	  333548 B/op	     959 allocs/op
+BenchmarkCodecXGenSuite/options-false.../Benchmark__JsonIter___Encode-8         	    3000	    476215 ns/op	  183552 B/op	    3262 allocs/op
+BenchmarkCodecXGenSuite/options-false.../Benchmark__Bson_______Encode-8         	    2000	    741688 ns/op	  715539 B/op	    5629 allocs/op
+BenchmarkCodecXGenSuite/options-false.../Benchmark__VMsgpack___Encode-8         	    2000	    649516 ns/op	  320385 B/op	     542 allocs/op
+BenchmarkCodecXGenSuite/options-false.../Benchmark__Msgp_______Encode-8         	   30000	     57573 ns/op	       0 B/op	       0 allocs/op
+BenchmarkCodecXGenSuite/options-false.../Benchmark__Easyjson___Encode-8         	    5000	    366701 ns/op	   92762 B/op	      14 allocs/op
+BenchmarkCodecXGenSuite/options-false.../Benchmark__Ffjson_____Encode-8         	    3000	    568665 ns/op	  219803 B/op	    1569 allocs/op
+BenchmarkCodecXGenSuite/options-false.../Benchmark__Sereal_____Encode-8         	    5000	    365595 ns/op	  296303 B/op	    4285 allocs/op
+-------------------------------
+BenchmarkCodecXGenSuite/options-false.../Benchmark__Msgpack____Decode-8         	   10000	    244013 ns/op	  131912 B/op	    1112 allocs/op
+BenchmarkCodecXGenSuite/options-false.../Benchmark__Binc_______Decode-8         	    5000	    280478 ns/op	  131944 B/op	    1112 allocs/op
+BenchmarkCodecXGenSuite/options-false.../Benchmark__Simple_____Decode-8         	    5000	    247863 ns/op	  131912 B/op	    1112 allocs/op
+BenchmarkCodecXGenSuite/options-false.../Benchmark__Cbor_______Decode-8         	   10000	    244624 ns/op	  131912 B/op	    1112 allocs/op
+BenchmarkCodecXGenSuite/options-false.../Benchmark__Json_______Decode-8         	    3000	    571572 ns/op	  170824 B/op	    1376 allocs/op
+BenchmarkCodecXGenSuite/options-false.../Benchmark__Std_Json___Decode-8         	    1000	   2224320 ns/op	  276337 B/op	    6959 allocs/op
+BenchmarkCodecXGenSuite/options-false.../Benchmark__Gob________Decode-8         	    5000	    387137 ns/op	  256683 B/op	    3261 allocs/op
+BenchmarkCodecXGenSuite/options-false.../Benchmark__JsonIter___Decode-8         	    2000	    913324 ns/op	  301472 B/op	    7769 allocs/op
+BenchmarkCodecXGenSuite/options-false.../Benchmark__Bson_______Decode-8         	    2000	   1139852 ns/op	  373121 B/op	   15703 allocs/op
+BenchmarkCodecXGenSuite/options-false.../Benchmark__Msgp_______Decode-8         	   10000	    124270 ns/op	  112688 B/op	    1058 allocs/op
+BenchmarkCodecXGenSuite/options-false.../Benchmark__Easyjson___Decode-8         	    3000	    521070 ns/op	  184176 B/op	    1371 allocs/op
+BenchmarkCodecXGenSuite/options-false.../Benchmark__Ffjson_____Decode-8         	    2000	    970256 ns/op	  161798 B/op	    1927 allocs/op
+```

codec/bench/bench.sh

@@ -0,0 +1,107 @@
+#!/bin/bash
+
+# download the code and all its dependencies 
+_go_get() {
+    go get -u \
+       "github.com/ugorji/go/codec" "github.com/ugorji/go/codec"/codecgen \
+       github.com/tinylib/msgp/msgp github.com/tinylib/msgp \
+       github.com/pquerna/ffjson/ffjson github.com/pquerna/ffjson \
+       github.com/Sereal/Sereal/Go/sereal \
+       bitbucket.org/bodhisnarkva/cbor/go \
+       github.com/davecgh/go-xdr/xdr2 \
+       gopkg.in/mgo.v2/bson \
+       gopkg.in/vmihailenco/msgpack.v2 \
+       github.com/json-iterator/go \
+       github.com/mailru/easyjson/...
+}
+
+# add generated tag to the top of each file
+_prependbt() {
+    cat > ${2} <<EOF
+// +build generated
+
+EOF
+    cat ${1} >> ${2}
+    rm -f ${1}
+}
+
+# To run the full suite of benchmarks, including executing against the external frameworks
+# listed above, you MUST first run code generation for the frameworks that support it.
+#
+# If you want to run the benchmarks against code generated values.
+# Then first generate the code generated values from values_test.go named typed.
+# we cannot normally read a _test.go file, so temporarily copy it into a readable file.
+_gen() {
+    local zsfx="_generated_test.go"
+    # local z=`pwd`
+    # z=${z%%/src/*}
+    # Note: ensure you run the codecgen for this codebase
+    cp values_test.go v.go &&
+        echo "codecgen ..." &&
+        codecgen -nx -rt codecgen -t 'codecgen generated' -o values_codecgen${zsfx} -d 19780 v.go &&
+        echo "msgp ... " &&
+        msgp -unexported -tests=false -o=m9.go -file=v.go &&
+        _prependbt m9.go values_msgp${zsfx} &&
+        echo "easyjson ... " &&
+        easyjson -all -no_std_marshalers -omit_empty -output_filename e9.go v.go &&
+        _prependbt e9.go values_easyjson${zsfx} &&
+        echo "ffjson ... " && 
+        ffjson -force-regenerate -reset-fields -w f9.go v.go &&
+        _prependbt f9.go values_ffjson${zsfx} &&
+        sed -i '' -e 's+ MarshalJSON(+ _MarshalJSON(+g' values_ffjson${zsfx} &&
+        sed -i '' -e 's+ UnmarshalJSON(+ _UnmarshalJSON(+g' values_ffjson${zsfx} &&
+        rm -f easyjson-bootstrap*.go ffjson-inception* &&
+        rm -f v.go &&
+        echo "... DONE"
+}
+
+# run the full suite of tests
+#
+# Basically, its a sequence of
+# go test -tags "alltests x safe codecgen generated" -bench "CodecSuite or AllSuite or XSuite" -benchmem
+# 
+_suite() {
+    local t="alltests x"
+    local a=( "" "safe"  "notfastpath" "notfastpath safe" "codecgen" "codecgen safe")
+    local b=( "generated" "generated safe")
+    for i in "${a[@]}"
+    do
+        echo ">>>> bench TAGS: '$t $i' SUITE: BenchmarkCodecXSuite"
+        go test -run Nothing -tags "$t $i" -bench BenchmarkCodecXSuite -benchmem "$@"
+    done
+    for i in "${b[@]}"
+    do
+        echo ">>>> bench TAGS: '$t $i' SUITE: BenchmarkCodecXGenSuite"
+        go test -run Nothing -tags "$t $i" -bench BenchmarkCodecXGenSuite -benchmem "$@"
+    done
+}
+
+_usage() {
+    echo "usage: bench.sh -[dcs] for [download, code-generate and suite-of-tests] respectively"
+}
+
+_main() {
+    if [[ "$1" == "" ]]
+    then
+        _usage
+        return 1
+    fi
+    while getopts "dcs" flag
+    do
+        case "x$flag" in
+            'xd') shift; _go_get "$@" ;;
+            'xc') shift; _gen "$@" ;;
+            'xs') shift; _suite "$@" ;;
+            *) shift; _usage; return 1 ;;
+        esac
+    done
+    # shift $((OPTIND-1))
+}
+
+if [ "." = `dirname $0` ]
+then
+    _main "$@"
+else
+    echo "bench.sh must be run from the directory it resides in"
+    _usage
+fi 

codec/bench/bench_test.go

@@ -0,0 +1,379 @@
+// Copyright (c) 2012-2018 Ugorji Nwoke. All rights reserved.
+// Use of this source code is governed by a MIT license found in the LICENSE file.
+
+package codec
+
+import (
+	"bytes"
+	"encoding/gob"
+	"encoding/json"
+	"encoding/xml"
+	"reflect"
+	"runtime"
+	"testing"
+	"time"
+)
+
+// Sample way to run:
+// go test -bi -bv -bd=1 -benchmem -bench=.
+
+func init() {
+	testPreInitFns = append(testPreInitFns, benchPreInit)
+	testPostInitFns = append(testPostInitFns, benchPostInit)
+}
+
+var (
+	benchTs *TestStruc
+
+	approxSize int
+
+	benchCheckers []benchChecker
+)
+
+type benchEncFn func(interface{}, []byte) ([]byte, error)
+type benchDecFn func([]byte, interface{}) error
+type benchIntfFn func() interface{}
+
+type benchChecker struct {
+	name     string
+	encodefn benchEncFn
+	decodefn benchDecFn
+}
+
+func benchReinit() {
+	benchCheckers = nil
+}
+
+func benchPreInit() {
+	benchTs = newTestStruc(benchDepth, testNumRepeatString, true, !testSkipIntf, benchMapStringKeyOnly)
+	approxSize = approxDataSize(reflect.ValueOf(benchTs)) * 3 / 2 // multiply by 1.5 to appease msgp, and prevent alloc
+	// bytesLen := 1024 * 4 * (benchDepth + 1) * (benchDepth + 1)
+	// if bytesLen < approxSize {
+	// 	bytesLen = approxSize
+	// }
+
+	benchCheckers = append(benchCheckers,
+		benchChecker{"msgpack", fnMsgpackEncodeFn, fnMsgpackDecodeFn},
+		benchChecker{"binc", fnBincEncodeFn, fnBincDecodeFn},
+		benchChecker{"simple", fnSimpleEncodeFn, fnSimpleDecodeFn},
+		benchChecker{"cbor", fnCborEncodeFn, fnCborDecodeFn},
+		benchChecker{"json", fnJsonEncodeFn, fnJsonDecodeFn},
+		benchChecker{"std-json", fnStdJsonEncodeFn, fnStdJsonDecodeFn},
+		benchChecker{"gob", fnGobEncodeFn, fnGobDecodeFn},
+		benchChecker{"std-xml", fnStdXmlEncodeFn, fnStdXmlDecodeFn},
+	)
+}
+
+func benchPostInit() {
+	if benchDoInitBench {
+		runBenchInit()
+	}
+}
+
+func runBenchInit() {
+	// logT(nil, "..............................................")
+	logT(nil, "BENCHMARK INIT: %v", time.Now())
+	// logT(nil, "To run full benchmark comparing encodings, use: \"go test -bench=.\"")
+	logT(nil, "Benchmark: ")
+	logT(nil, "\tStruct recursive Depth:             %d", benchDepth)
+	if approxSize > 0 {
+		logT(nil, "\tApproxDeepSize Of benchmark Struct: %d bytes", approxSize)
+	}
+	if benchUnscientificRes {
+		logT(nil, "Benchmark One-Pass Run (with Unscientific Encode/Decode times): ")
+	} else {
+		logT(nil, "Benchmark One-Pass Run:")
+	}
+	for _, bc := range benchCheckers {
+		doBenchCheck(bc.name, bc.encodefn, bc.decodefn)
+	}
+	logT(nil, "..............................................")
+	if benchInitDebug {
+		logT(nil, "<<<<====>>>> depth: %v, ts: %#v\n", benchDepth, benchTs)
+	}
+	runtime.GC()
+	time.Sleep(100 * time.Millisecond)
+}
+
+var vBenchTs = TestStruc{}
+
+func fnBenchNewTs() interface{} {
+	vBenchTs = TestStruc{}
+	return &vBenchTs
+	// return new(TestStruc)
+}
+
+// const benchCheckDoDeepEqual = false
+
+func benchRecoverPanic(t interface{}) {
+	if r := recover(); r != nil {
+		logT(t, "panic: %v\n", r)
+	}
+}
+
+func doBenchCheck(name string, encfn benchEncFn, decfn benchDecFn) {
+	// if benchUnscientificRes {
+	// 	logT(nil, "-------------- %s ----------------", name)
+	// }
+	defer benchRecoverPanic(nil)
+	runtime.GC()
+	tnow := time.Now()
+	buf, err := encfn(benchTs, nil)
+	if err != nil {
+		logT(nil, "\t%10s: **** Error encoding benchTs: %v", name, err)
+		return
+	}
+	encDur := time.Since(tnow)
+	encLen := len(buf)
+	runtime.GC()
+	if !benchUnscientificRes {
+		logT(nil, "\t%10s: len: %d bytes\n", name, encLen)
+		return
+	}
+	tnow = time.Now()
+	var ts2 TestStruc
+	if err = decfn(buf, &ts2); err != nil {
+		logT(nil, "\t%10s: **** Error decoding into new TestStruc: %v", name, err)
+		return
+	}
+	decDur := time.Since(tnow)
+	// if benchCheckDoDeepEqual {
+	if benchVerify {
+		err = deepEqual(benchTs, &ts2)
+		if err == nil {
+			logT(nil, "\t%10s: len: %d bytes,\t encode: %v,\t decode: %v,\tencoded = decoded", name, encLen, encDur, decDur)
+		} else {
+			logT(nil, "\t%10s: len: %d bytes,\t encode: %v,\t decode: %v,\tencoded != decoded: %v", name, encLen, encDur, decDur, err)
+			// if strings.Contains(name, "json") {
+			// 	println(">>>>>")
+			// 	f1, _ := os.Create("1.out")
+			// 	f2, _ := os.Create("2.out")
+			// 	f3, _ := os.Create("3.json")
+			// 	buf3, _ := json.MarshalIndent(&ts2, "", "\t")
+			// 	spew.Config.SortKeys = true
+			// 	spew.Config.SpewKeys = true
+			// 	println("^^^^^^^^^^^^^^")
+			// 	spew.Fdump(f1, benchTs)
+			// 	println("^^^^^^^^^^^^^^")
+			// 	spew.Fdump(f2, &ts2)
+			// 	println("^^^^^^^^^^^^^^")
+			// 	f3.Write(buf3)
+			// 	f1.Close()
+			// 	f2.Close()
+			// 	f3.Close()
+			// }
+			// logT(nil, "\t: err: %v,\n benchTs: %#v\n\n, ts2: %#v\n\n", err, benchTs, ts2) // TODO: remove
+			// logT(nil, "BenchVerify: Error comparing en|decoded TestStruc: %v", err)
+			// return
+			// logT(nil, "BenchVerify: Error comparing benchTs: %v\n--------\n%v\n--------\n%v", err, benchTs, ts2)
+			// if strings.Contains(name, "json") {
+			// 	logT(nil, "\n\tDECODED FROM\n--------\n%s", buf)
+			// }
+		}
+	} else {
+		logT(nil, "\t%10s: len: %d bytes,\t encode: %v,\t decode: %v", name, encLen, encDur, decDur)
+	}
+	return
+}
+
+func fnBenchmarkEncode(b *testing.B, encName string, ts interface{}, encfn benchEncFn) {
+	defer benchRecoverPanic(b)
+	testOnce.Do(testInitAll)
+	var err error
+	bs := make([]byte, 0, approxSize)
+	runtime.GC()
+	b.ResetTimer()
+	for i := 0; i < b.N; i++ {
+		if _, err = encfn(ts, bs); err != nil {
+			break
+		}
+	}
+	if err != nil {
+		logT(b, "Error encoding benchTs: %s: %v", encName, err)
+		b.FailNow()
+	}
+}
+
+func fnBenchmarkDecode(b *testing.B, encName string, ts interface{},
+	encfn benchEncFn, decfn benchDecFn, newfn benchIntfFn,
+) {
+	defer benchRecoverPanic(b)
+	testOnce.Do(testInitAll)
+	bs := make([]byte, 0, approxSize)
+	buf, err := encfn(ts, bs)
+	if err != nil {
+		logT(b, "Error encoding benchTs: %s: %v", encName, err)
+		b.FailNow()
+	}
+	if false && benchVerify { // do not do benchVerify during decode
+		// ts2 := newfn()
+		ts1 := ts.(*TestStruc)
+		ts2 := new(TestStruc)
+		if err = decfn(buf, ts2); err != nil {
+			logT(b, "BenchVerify: Error decoding benchTs: %s: %v", encName, err)
+			b.FailNow()
+		}
+		if err = deepEqual(ts1, ts2); err != nil {
+			logT(b, "BenchVerify: Error comparing benchTs: %s: %v", encName, err)
+			b.FailNow()
+		}
+	}
+	runtime.GC()
+	b.ResetTimer()
+	for i := 0; i < b.N; i++ {
+		ts = newfn()
+		if err = decfn(buf, ts); err != nil {
+			break
+		}
+	}
+	if err != nil {
+		logT(b, "Error decoding into new TestStruc: %s: %v", encName, err)
+		b.FailNow()
+	}
+}
+
+// ------------ tests below
+
+func fnMsgpackEncodeFn(ts interface{}, bsIn []byte) (bs []byte, err error) {
+	return sTestCodecEncode(ts, bsIn, fnBenchmarkByteBuf, testMsgpackH, &testMsgpackH.BasicHandle)
+}
+
+func fnMsgpackDecodeFn(buf []byte, ts interface{}) error {
+	return sTestCodecDecode(buf, ts, testMsgpackH, &testMsgpackH.BasicHandle)
+}
+
+func fnBincEncodeFn(ts interface{}, bsIn []byte) (bs []byte, err error) {
+	return sTestCodecEncode(ts, bsIn, fnBenchmarkByteBuf, testBincH, &testBincH.BasicHandle)
+}
+
+func fnBincDecodeFn(buf []byte, ts interface{}) error {
+	return sTestCodecDecode(buf, ts, testBincH, &testBincH.BasicHandle)
+}
+
+func fnSimpleEncodeFn(ts interface{}, bsIn []byte) (bs []byte, err error) {
+	return sTestCodecEncode(ts, bsIn, fnBenchmarkByteBuf, testSimpleH, &testSimpleH.BasicHandle)
+}
+
+func fnSimpleDecodeFn(buf []byte, ts interface{}) error {
+	return sTestCodecDecode(buf, ts, testSimpleH, &testSimpleH.BasicHandle)
+}
+
+func fnCborEncodeFn(ts interface{}, bsIn []byte) (bs []byte, err error) {
+	return sTestCodecEncode(ts, bsIn, fnBenchmarkByteBuf, testCborH, &testCborH.BasicHandle)
+}
+
+func fnCborDecodeFn(buf []byte, ts interface{}) error {
+	return sTestCodecDecode(buf, ts, testCborH, &testCborH.BasicHandle)
+}
+
+func fnJsonEncodeFn(ts interface{}, bsIn []byte) (bs []byte, err error) {
+	return sTestCodecEncode(ts, bsIn, fnBenchmarkByteBuf, testJsonH, &testJsonH.BasicHandle)
+}
+
+func fnJsonDecodeFn(buf []byte, ts interface{}) error {
+	return sTestCodecDecode(buf, ts, testJsonH, &testJsonH.BasicHandle)
+}
+
+func fnGobEncodeFn(ts interface{}, bsIn []byte) ([]byte, error) {
+	buf := fnBenchmarkByteBuf(bsIn)
+	err := gob.NewEncoder(buf).Encode(ts)
+	return buf.Bytes(), err
+}
+
+func fnGobDecodeFn(buf []byte, ts interface{}) error {
+	return gob.NewDecoder(bytes.NewReader(buf)).Decode(ts)
+}
+
+func fnStdXmlEncodeFn(ts interface{}, bsIn []byte) ([]byte, error) {
+	buf := fnBenchmarkByteBuf(bsIn)
+	err := xml.NewEncoder(buf).Encode(ts)
+	return buf.Bytes(), err
+}
+
+func fnStdXmlDecodeFn(buf []byte, ts interface{}) error {
+	return xml.NewDecoder(bytes.NewReader(buf)).Decode(ts)
+}
+
+func fnStdJsonEncodeFn(ts interface{}, bsIn []byte) ([]byte, error) {
+	if testUseIoEncDec >= 0 {
+		buf := fnBenchmarkByteBuf(bsIn)
+		err := json.NewEncoder(buf).Encode(ts)
+		return buf.Bytes(), err
+	}
+	return json.Marshal(ts)
+}
+
+func fnStdJsonDecodeFn(buf []byte, ts interface{}) error {
+	if testUseIoEncDec >= 0 {
+		return json.NewDecoder(bytes.NewReader(buf)).Decode(ts)
+	}
+	return json.Unmarshal(buf, ts)
+}
+
+// ----------- DECODE ------------------
+
+func Benchmark__Msgpack____Encode(b *testing.B) {
+	fnBenchmarkEncode(b, "msgpack", benchTs, fnMsgpackEncodeFn)
+}
+
+func Benchmark__Binc_______Encode(b *testing.B) {
+	fnBenchmarkEncode(b, "binc", benchTs, fnBincEncodeFn)
+}
+
+func Benchmark__Simple_____Encode(b *testing.B) {
+	fnBenchmarkEncode(b, "simple", benchTs, fnSimpleEncodeFn)
+}
+
+func Benchmark__Cbor_______Encode(b *testing.B) {
+	fnBenchmarkEncode(b, "cbor", benchTs, fnCborEncodeFn)
+}
+
+func Benchmark__Json_______Encode(b *testing.B) {
+	fnBenchmarkEncode(b, "json", benchTs, fnJsonEncodeFn)
+}
+
+func Benchmark__Std_Json___Encode(b *testing.B) {
+	fnBenchmarkEncode(b, "std-json", benchTs, fnStdJsonEncodeFn)
+}
+
+func Benchmark__Gob________Encode(b *testing.B) {
+	fnBenchmarkEncode(b, "gob", benchTs, fnGobEncodeFn)
+}
+
+func Benchmark__Std_Xml____Encode(b *testing.B) {
+	fnBenchmarkEncode(b, "std-xml", benchTs, fnStdXmlEncodeFn)
+}
+
+// ----------- DECODE ------------------
+
+func Benchmark__Msgpack____Decode(b *testing.B) {
+	fnBenchmarkDecode(b, "msgpack", benchTs, fnMsgpackEncodeFn, fnMsgpackDecodeFn, fnBenchNewTs)
+}
+
+func Benchmark__Binc_______Decode(b *testing.B) {
+	fnBenchmarkDecode(b, "binc", benchTs, fnBincEncodeFn, fnBincDecodeFn, fnBenchNewTs)
+}
+
+func Benchmark__Simple_____Decode(b *testing.B) {
+	fnBenchmarkDecode(b, "simple", benchTs, fnSimpleEncodeFn, fnSimpleDecodeFn, fnBenchNewTs)
+}
+
+func Benchmark__Cbor_______Decode(b *testing.B) {
+	fnBenchmarkDecode(b, "cbor", benchTs, fnCborEncodeFn, fnCborDecodeFn, fnBenchNewTs)
+}
+
+func Benchmark__Json_______Decode(b *testing.B) {
+	fnBenchmarkDecode(b, "json", benchTs, fnJsonEncodeFn, fnJsonDecodeFn, fnBenchNewTs)
+}
+
+func Benchmark__Std_Json___Decode(b *testing.B) {
+	fnBenchmarkDecode(b, "std-json", benchTs, fnStdJsonEncodeFn, fnStdJsonDecodeFn, fnBenchNewTs)
+}
+
+func Benchmark__Gob________Decode(b *testing.B) {
+	fnBenchmarkDecode(b, "gob", benchTs, fnGobEncodeFn, fnGobDecodeFn, fnBenchNewTs)
+}
+
+func Benchmark__Std_Xml____Decode(b *testing.B) {
+	fnBenchmarkDecode(b, "std-xml", benchTs, fnStdXmlEncodeFn, fnStdXmlDecodeFn, fnBenchNewTs)
+}

codec/bench/doc.go

@@ -0,0 +1,14 @@
+// Copyright (c) 2012-2018 Ugorji Nwoke. All rights reserved.
+// Use of this source code is governed by a MIT license found in the LICENSE file.
+
+/*
+Package codec provides a
+High Performance, Feature-Rich Idiomatic Go 1.4+ codec/encoding library
+for binc, msgpack, cbor, json.
+
+Here, we have the benchmark files comparing against other encoding libraries.
+
+See README.md for more details.
+*/
+package codec 
+

codec/bench/shared_test.go

@@ -0,0 +1,316 @@
+// Copyright (c) 2012-2018 Ugorji Nwoke. All rights reserved.
+// Use of this source code is governed by a MIT license found in the LICENSE file.
+
+package codec
+
+// This file sets up the variables used, including testInitFns.
+// Each file should add initialization that should be performed
+// after flags are parsed.
+//
+// init is a multi-step process:
+//   - setup vars (handled by init functions in each file)
+//   - parse flags
+//   - setup derived vars (handled by pre-init registered functions - registered in init function)
+//   - post init (handled by post-init registered functions - registered in init function)
+// This way, no one has to manage carefully control the initialization
+// using file names, etc.
+//
+// Tests which require external dependencies need the -tag=x parameter.
+// They should be run as:
+//    go test -tags=x -run=. <other parameters ...>
+// Benchmarks should also take this parameter, to include the sereal, xdr, etc.
+// To run against codecgen, etc, make sure you pass extra parameters.
+// Example usage:
+//    go test "-tags=x codecgen" -bench=. <other parameters ...>
+//
+// To fully test everything:
+//    go test -tags=x -benchtime=100ms -tv -bg -bi  -brw -bu -v -run=. -bench=.
+
+// Handling flags
+// codec_test.go will define a set of global flags for testing, including:
+//   - Use Reset
+//   - Use IO reader/writer (vs direct bytes)
+//   - Set Canonical
+//   - Set InternStrings
+//   - Use Symbols
+//
+// This way, we can test them all by running same set of tests with a different
+// set of flags.
+//
+// Following this, all the benchmarks will utilize flags set by codec_test.go
+// and will not redefine these "global" flags.
+
+import (
+	"bytes"
+	"flag"
+	"fmt"
+	"io"
+	"io/ioutil"
+	"log"
+	"sync"
+	"testing"
+)
+
+import . "github.com/ugorji/go/codec"
+
+type testHED struct {
+	H Handle
+	E *Encoder
+	D *Decoder
+}
+
+type ioReaderWrapper struct {
+	r io.Reader
+}
+
+func (x ioReaderWrapper) Read(p []byte) (n int, err error) {
+	return x.r.Read(p)
+}
+
+type ioWriterWrapper struct {
+	w io.Writer
+}
+
+func (x ioWriterWrapper) Write(p []byte) (n int, err error) {
+	return x.w.Write(p)
+}
+
+var (
+	// testNoopH    = NoopHandle(8)
+	testMsgpackH = &MsgpackHandle{}
+	testBincH    = &BincHandle{}
+	testSimpleH  = &SimpleHandle{}
+	testCborH    = &CborHandle{}
+	testJsonH    = &JsonHandle{}
+
+	testHandles     []Handle
+	testPreInitFns  []func()
+	testPostInitFns []func()
+
+	testOnce sync.Once
+
+	testHEDs []testHED
+)
+
+// flag variables used by tests (and bench)
+var (
+	testDepth int
+
+	testVerbose       bool
+	testInitDebug     bool
+	testStructToArray bool
+	testCanonical     bool
+	testUseReset      bool
+	testSkipIntf      bool
+	testInternStr     bool
+	testUseMust       bool
+	testCheckCircRef  bool
+
+	testUseIoEncDec  int
+	testUseIoWrapper bool
+
+	testMaxInitLen int
+
+	testNumRepeatString int
+
+	testRpcBufsize int
+)
+
+// variables that are not flags, but which can configure the handles
+var (
+	testEncodeOptions EncodeOptions
+	testDecodeOptions DecodeOptions
+)
+
+// flag variables used by bench
+var (
+	benchDoInitBench      bool
+	benchVerify           bool
+	benchUnscientificRes  bool = false
+	benchMapStringKeyOnly bool
+	//depth of 0 maps to ~400bytes json-encoded string, 1 maps to ~1400 bytes, etc
+	//For depth>1, we likely trigger stack growth for encoders, making benchmarking unreliable.
+	benchDepth     int
+	benchInitDebug bool
+)
+
+func init() {
+	log.SetOutput(ioutil.Discard) // don't allow things log to standard out/err
+	testHEDs = make([]testHED, 0, 32)
+	testHandles = append(testHandles,
+		// testNoopH,
+		testMsgpackH, testBincH, testSimpleH, testCborH, testJsonH)
+	// set ExplicitRelease on each handle
+	testMsgpackH.ExplicitRelease = true
+	testBincH.ExplicitRelease = true
+	testSimpleH.ExplicitRelease = true
+	testCborH.ExplicitRelease = true
+	testJsonH.ExplicitRelease = true
+
+	testInitFlags()
+	benchInitFlags()
+}
+
+func testInitFlags() {
+	// delete(testDecOpts.ExtFuncs, timeTyp)
+	flag.IntVar(&testDepth, "tsd", 0, "Test Struc Depth")
+	flag.BoolVar(&testVerbose, "tv", false, "Test Verbose (no longer used - here for compatibility)")
+	flag.BoolVar(&testInitDebug, "tg", false, "Test Init Debug")
+	flag.IntVar(&testUseIoEncDec, "ti", -1, "Use IO Reader/Writer for Marshal/Unmarshal ie >= 0")
+	flag.BoolVar(&testUseIoWrapper, "tiw", false, "Wrap the IO Reader/Writer with a base pass-through reader/writer")
+	flag.BoolVar(&testStructToArray, "ts", false, "Set StructToArray option")
+	flag.BoolVar(&testCanonical, "tc", false, "Set Canonical option")
+	flag.BoolVar(&testInternStr, "te", false, "Set InternStr option")
+	flag.BoolVar(&testSkipIntf, "tf", false, "Skip Interfaces")
+	flag.BoolVar(&testUseReset, "tr", false, "Use Reset")
+	flag.IntVar(&testNumRepeatString, "trs", 8, "Create string variables by repeating a string N times")
+	flag.IntVar(&testMaxInitLen, "tx", 0, "Max Init Len")
+	flag.BoolVar(&testUseMust, "tm", true, "Use Must(En|De)code")
+	flag.BoolVar(&testCheckCircRef, "tl", false, "Use Check Circular Ref")
+}
+
+func benchInitFlags() {
+	flag.BoolVar(&benchMapStringKeyOnly, "bs", false, "Bench use maps with string keys only")
+	flag.BoolVar(&benchInitDebug, "bg", false, "Bench Debug")
+	flag.IntVar(&benchDepth, "bd", 1, "Bench Depth")
+	flag.BoolVar(&benchDoInitBench, "bi", false, "Run Bench Init")
+	flag.BoolVar(&benchVerify, "bv", false, "Verify Decoded Value during Benchmark")
+	flag.BoolVar(&benchUnscientificRes, "bu", false, "Show Unscientific Results during Benchmark")
+}
+
+func testHEDGet(h Handle) *testHED {
+	for i := range testHEDs {
+		v := &testHEDs[i]
+		if v.H == h {
+			return v
+		}
+	}
+	testHEDs = append(testHEDs, testHED{h, NewEncoder(nil, h), NewDecoder(nil, h)})
+	return &testHEDs[len(testHEDs)-1]
+}
+
+func testReinit() {
+	testOnce = sync.Once{}
+	testHEDs = nil
+}
+
+func testInitAll() {
+	// only parse it once.
+	if !flag.Parsed() {
+		flag.Parse()
+	}
+	for _, f := range testPreInitFns {
+		f()
+	}
+	for _, f := range testPostInitFns {
+		f()
+	}
+}
+
+func sTestCodecEncode(ts interface{}, bsIn []byte, fn func([]byte) *bytes.Buffer,
+	h Handle, bh *BasicHandle) (bs []byte, err error) {
+	// bs = make([]byte, 0, approxSize)
+	var e *Encoder
+	var buf *bytes.Buffer
+	if testUseReset {
+		e = testHEDGet(h).E
+	} else {
+		e = NewEncoder(nil, h)
+	}
+	var oldWriteBufferSize int
+	if testUseIoEncDec >= 0 {
+		buf = fn(bsIn)
+		// set the encode options for using a buffer
+		oldWriteBufferSize = bh.WriterBufferSize
+		bh.WriterBufferSize = testUseIoEncDec
+		if testUseIoWrapper {
+			e.Reset(ioWriterWrapper{buf})
+		} else {
+			e.Reset(buf)
+		}
+	} else {
+		bs = bsIn
+		e.ResetBytes(&bs)
+	}
+	if testUseMust {
+		e.MustEncode(ts)
+	} else {
+		err = e.Encode(ts)
+	}
+	if testUseIoEncDec >= 0 {
+		bs = buf.Bytes()
+		bh.WriterBufferSize = oldWriteBufferSize
+	}
+	if !testUseReset {
+		e.Release()
+	}
+	return
+}
+
+func sTestCodecDecode(bs []byte, ts interface{}, h Handle, bh *BasicHandle) (err error) {
+	var d *Decoder
+	// var buf *bytes.Reader
+	if testUseReset {
+		d = testHEDGet(h).D
+	} else {
+		d = NewDecoder(nil, h)
+	}
+	var oldReadBufferSize int
+	if testUseIoEncDec >= 0 {
+		buf := bytes.NewReader(bs)
+		oldReadBufferSize = bh.ReaderBufferSize
+		bh.ReaderBufferSize = testUseIoEncDec
+		if testUseIoWrapper {
+			d.Reset(ioReaderWrapper{buf})
+		} else {
+			d.Reset(buf)
+		}
+	} else {
+		d.ResetBytes(bs)
+	}
+	if testUseMust {
+		d.MustDecode(ts)
+	} else {
+		err = d.Decode(ts)
+	}
+	if testUseIoEncDec >= 0 {
+		bh.ReaderBufferSize = oldReadBufferSize
+	}
+	if !testUseReset {
+		d.Release()
+	}
+	return
+}
+
+// --- functions below are used by both benchmarks and tests
+
+func logT(x interface{}, format string, args ...interface{}) {
+	if t, ok := x.(*testing.T); ok && t != nil {
+		t.Logf(format, args...)
+	} else if b, ok := x.(*testing.B); ok && b != nil {
+		b.Logf(format, args...)
+	} else { // if testing.Verbose() { // if testVerbose {
+		if len(format) == 0 || format[len(format)-1] != '\n' {
+			format = format + "\n"
+		}
+		fmt.Printf(format, args...)
+	}
+}
+
+// --- functions below are used only by benchmarks alone
+
+func fnBenchmarkByteBuf(bsIn []byte) (buf *bytes.Buffer) {
+	// var buf bytes.Buffer
+	// buf.Grow(approxSize)
+	buf = bytes.NewBuffer(bsIn)
+	buf.Truncate(0)
+	return
+}
+
+// func benchFnCodecEncode(ts interface{}, bsIn []byte, h Handle) (bs []byte, err error) {
+// 	return testCodecEncode(ts, bsIn, fnBenchmarkByteBuf, h)
+// }
+
+// func benchFnCodecDecode(bs []byte, ts interface{}, h Handle) (err error) {
+// 	return testCodecDecode(bs, ts, h)
+// }

codec/bench/values_test.go

@@ -0,0 +1,401 @@
+// comment this out // + build testing
+
+// Copyright (c) 2012-2018 Ugorji Nwoke. All rights reserved.
+// Use of this source code is governed by a MIT license found in the LICENSE file.
+
+package codec
+
+// This file contains values used by tests and benchmarks.
+// The benchmarks will test performance against other libraries
+// (encoding/json, json-iterator, bson, gob, etc).
+// Consequently, we only use values that will parse well in all engines,
+// and only leverage features that work across multiple libraries for a truer comparison.
+// For example,
+// - JSON/BSON do not like maps with keys that are not strings,
+//   so we only use maps with string keys here.
+// - _struct options are not honored by other libraries,
+//   so we don't use them in this file.
+
+import (
+	"math"
+	"strings"
+)
+
+// func init() {
+// 	rt := reflect.TypeOf((*TestStruc)(nil)).Elem()
+// 	defTypeInfos.get(rt2id(rt), rt)
+// }
+
+type wrapSliceUint64 []uint64
+type wrapSliceString []string
+type wrapUint64 uint64
+type wrapString string
+type wrapUint64Slice []wrapUint64
+type wrapStringSlice []wrapString
+
+type stringUint64T struct {
+	S string
+	U uint64
+}
+
+type AnonInTestStruc struct {
+	AS         string
+	AI64       int64
+	AI16       int16
+	AUi64      uint64
+	ASslice    []string
+	AI64slice  []int64
+	AUi64slice []uint64
+	AF64slice  []float64
+	AF32slice  []float32
+
+	// AMI32U32  map[int32]uint32
+	// AMU32F64 map[uint32]float64 // json/bson do not like it
+	AMSU16 map[string]uint16
+
+	// use these to test 0-len or nil slices/maps/arrays
+	AI64arr0    [0]int64
+	A164slice0  []int64
+	AUi64sliceN []uint64
+	AMSU16N     map[string]uint16
+	AMSU16E     map[string]uint16
+}
+
+// testSimpleFields is a sub-set of TestStrucCommon
+type testSimpleFields struct {
+	S string
+
+	I64 int64
+	I8  int8
+
+	Ui64 uint64
+	Ui8  uint8
+
+	F64 float64
+	F32 float32
+
+	B bool
+
+	Sslice    []string
+	I16slice  []int16
+	Ui64slice []uint64
+	Ui8slice  []uint8
+	Bslice    []bool
+
+	Iptrslice []*int64
+
+	WrapSliceInt64  wrapSliceUint64
+	WrapSliceString wrapSliceString
+
+	Msi64 map[string]int64
+}
+
+type TestStrucCommon struct {
+	S string
+
+	I64 int64
+	I32 int32
+	I16 int16
+	I8  int8
+
+	I64n int64
+	I32n int32
+	I16n int16
+	I8n  int8
+
+	Ui64 uint64
+	Ui32 uint32
+	Ui16 uint16
+	Ui8  uint8
+
+	F64 float64
+	F32 float32
+
+	B  bool
+	By uint8 // byte: msgp doesn't like byte
+
+	Sslice    []string
+	I64slice  []int64
+	I16slice  []int16
+	Ui64slice []uint64
+	Ui8slice  []uint8
+	Bslice    []bool
+	Byslice   []byte
+
+	Iptrslice []*int64
+
+	WrapSliceInt64  wrapSliceUint64
+	WrapSliceString wrapSliceString
+
+	Msi64 map[string]int64
+
+	Simplef testSimpleFields
+
+	SstrUi64T []stringUint64T
+
+	AnonInTestStruc
+
+	NotAnon AnonInTestStruc
+
+	// R          Raw // Testing Raw must be explicitly turned on, so use standalone test
+	// Rext RawExt // Testing RawExt is tricky, so use standalone test
+
+	Nmap   map[string]bool //don't set this, so we can test for nil
+	Nslice []byte          //don't set this, so we can test for nil
+	Nint64 *int64          //don't set this, so we can test for nil
+}
+
+type TestStruc struct {
+	// _struct struct{} `json:",omitempty"` //set omitempty for every field
+
+	TestStrucCommon
+
+	Mtsptr     map[string]*TestStruc
+	Mts        map[string]TestStruc
+	Its        []*TestStruc
+	Nteststruc *TestStruc
+}
+
+func populateTestStrucCommon(ts *TestStrucCommon, n int, bench, useInterface, useStringKeyOnly bool) {
+	var i64a, i64b, i64c, i64d int64 = 64, 6464, 646464, 64646464
+
+	// if bench, do not use uint64 values > math.MaxInt64, as bson, etc cannot decode them
+
+	var a = AnonInTestStruc{
+		// There's more leeway in altering this.
+		AS:    strRpt(n, "A-String"),
+		AI64:  -64646464,
+		AI16:  1616,
+		AUi64: 64646464,
+		// (U+1D11E)G-clef character may be represented in json as "\uD834\uDD1E".
+		// single reverse solidus character may be represented in json as "\u005C".
+		// include these in ASslice below.
+		ASslice: []string{
+			strRpt(n, "Aone"),
+			strRpt(n, "Atwo"),
+			strRpt(n, "Athree"),
+			strRpt(n, "Afour.reverse_solidus.\u005c"),
+			strRpt(n, "Afive.Gclef.\U0001d11E\"ugorji\"done.")},
+		AI64slice: []int64{
+			0, 1, -1, -22, 333, -4444, 55555, -666666,
+			// msgpack ones
+			-48, -32, -24, -8, 32, 127, 192, 255,
+			// standard ones
+			0, -1, 1,
+			math.MaxInt8, math.MaxInt8 + 4, math.MaxInt8 - 4,
+			math.MaxInt16, math.MaxInt16 + 4, math.MaxInt16 - 4,
+			math.MaxInt32, math.MaxInt32 + 4, math.MaxInt32 - 4,
+			math.MaxInt64, math.MaxInt64 - 4,
+			math.MinInt8, math.MinInt8 + 4, math.MinInt8 - 4,
+			math.MinInt16, math.MinInt16 + 4, math.MinInt16 - 4,
+			math.MinInt32, math.MinInt32 + 4, math.MinInt32 - 4,
+			math.MinInt64, math.MinInt64 + 4,
+		},
+		AUi64slice: []uint64{
+			0, 1, 22, 333, 4444, 55555, 666666,
+			// standard ones
+			math.MaxUint8, math.MaxUint8 + 4, math.MaxUint8 - 4,
+			math.MaxUint16, math.MaxUint16 + 4, math.MaxUint16 - 4,
+			math.MaxUint32, math.MaxUint32 + 4, math.MaxUint32 - 4,
+		},
+		AMSU16: map[string]uint16{strRpt(n, "1"): 1, strRpt(n, "22"): 2, strRpt(n, "333"): 3, strRpt(n, "4444"): 4},
+
+		// Note: +/- inf, NaN, and other non-representable numbers should not be explicitly tested here
+
+		AF64slice: []float64{
+			11.11e-11, -11.11e+11,
+			2.222E+12, -2.222E-12,
+			-555.55E-5, 555.55E+5,
+			666.66E-6, -666.66E+6,
+			7777.7777E-7, -7777.7777E-7,
+			-8888.8888E+8, 8888.8888E+8,
+			-99999.9999E+9, 99999.9999E+9,
+			// these below are hairy enough to need strconv.ParseFloat
+			33.33E-33, -33.33E+33,
+			44.44e+44, -44.44e-44,
+			// standard ones
+			0, -1, 1,
+			// math.Inf(1), math.Inf(-1),
+			math.Pi, math.Phi, math.E,
+			math.MaxFloat64, math.SmallestNonzeroFloat64,
+		},
+		AF32slice: []float32{
+			11.11e-11, -11.11e+11,
+			2.222E+12, -2.222E-12,
+			-555.55E-5, 555.55E+5,
+			666.66E-6, -666.66E+6,
+			7777.7777E-7, -7777.7777E-7,
+			-8888.8888E+8, 8888.8888E+8,
+			-99999.9999E+9, 99999.9999E+9,
+			// these below are hairy enough to need strconv.ParseFloat
+			33.33E-33, -33.33E+33,
+			// standard ones
+			0, -1, 1,
+			// math.Float32frombits(0x7FF00000), math.Float32frombits(0xFFF00000), //+inf and -inf
+			math.MaxFloat32, math.SmallestNonzeroFloat32,
+		},
+
+		A164slice0:  []int64{},
+		AUi64sliceN: nil,
+		AMSU16N:     nil,
+		AMSU16E:     map[string]uint16{},
+	}
+
+	if !bench {
+		a.AUi64slice = append(a.AUi64slice, math.MaxUint64, math.MaxUint64-4)
+	}
+	*ts = TestStrucCommon{
+		S: strRpt(n, `some really really cool names that are nigerian and american like "ugorji melody nwoke" - get it? `),
+
+		// set the numbers close to the limits
+		I8:   math.MaxInt8 * 2 / 3,  // 8,
+		I8n:  math.MinInt8 * 2 / 3,  // 8,
+		I16:  math.MaxInt16 * 2 / 3, // 16,
+		I16n: math.MinInt16 * 2 / 3, // 16,
+		I32:  math.MaxInt32 * 2 / 3, // 32,
+		I32n: math.MinInt32 * 2 / 3, // 32,
+		I64:  math.MaxInt64 * 2 / 3, // 64,
+		I64n: math.MinInt64 * 2 / 3, // 64,
+
+		Ui64: math.MaxUint64 * 2 / 3, // 64
+		Ui32: math.MaxUint32 * 2 / 3, // 32
+		Ui16: math.MaxUint16 * 2 / 3, // 16
+		Ui8:  math.MaxUint8 * 2 / 3,  // 8
+
+		F32: 3.402823e+38, // max representable float32 without losing precision
+		F64: 3.40281991833838838338e+53,
+
+		B:  true,
+		By: 5,
+
+		Sslice:    []string{strRpt(n, "one"), strRpt(n, "two"), strRpt(n, "three")},
+		I64slice:  []int64{1111, 2222, 3333},
+		I16slice:  []int16{44, 55, 66},
+		Ui64slice: []uint64{12121212, 34343434, 56565656},
+		Ui8slice:  []uint8{210, 211, 212},
+		Bslice:    []bool{true, false, true, false},
+		Byslice:   []byte{13, 14, 15},
+
+		Msi64: map[string]int64{
+			strRpt(n, "one"):       1,
+			strRpt(n, "two"):       2,
+			strRpt(n, "\"three\""): 3,
+		},
+
+		WrapSliceInt64:  []uint64{4, 16, 64, 256},
+		WrapSliceString: []string{strRpt(n, "4"), strRpt(n, "16"), strRpt(n, "64"), strRpt(n, "256")},
+
+		// R: Raw([]byte("goodbye")),
+		// Rext: RawExt{ 120, []byte("hello"), }, // TODO: don't set this - it's hard to test
+
+		// DecodeNaked bombs here, because the stringUint64T is decoded as a map,
+		// and a map cannot be the key type of a map.
+		// Thus, don't initialize this here.
+		// Msu2wss: map[stringUint64T]wrapStringSlice{
+		// 	{"5", 5}: []wrapString{"1", "2", "3", "4", "5"},
+		// 	{"3", 3}: []wrapString{"1", "2", "3"},
+		// },
+
+		// make Simplef same as top-level
+		// TODO: should this have slightly different values???
+		Simplef: testSimpleFields{
+			S: strRpt(n, `some really really cool names that are nigerian and american like "ugorji melody nwoke" - get it? `),
+
+			// set the numbers close to the limits
+			I8:  math.MaxInt8 * 2 / 3,  // 8,
+			I64: math.MaxInt64 * 2 / 3, // 64,
+
+			Ui64: math.MaxUint64 * 2 / 3, // 64
+			Ui8:  math.MaxUint8 * 2 / 3,  // 8
+
+			F32: 3.402823e+38, // max representable float32 without losing precision
+			F64: 3.40281991833838838338e+53,
+
+			B: true,
+
+			Sslice:    []string{strRpt(n, "one"), strRpt(n, "two"), strRpt(n, "three")},
+			I16slice:  []int16{44, 55, 66},
+			Ui64slice: []uint64{12121212, 34343434, 56565656},
+			Ui8slice:  []uint8{210, 211, 212},
+			Bslice:    []bool{true, false, true, false},
+
+			Msi64: map[string]int64{
+				strRpt(n, "one"):       1,
+				strRpt(n, "two"):       2,
+				strRpt(n, "\"three\""): 3,
+			},
+
+			WrapSliceInt64:  []uint64{4, 16, 64, 256},
+			WrapSliceString: []string{strRpt(n, "4"), strRpt(n, "16"), strRpt(n, "64"), strRpt(n, "256")},
+		},
+
+		SstrUi64T:       []stringUint64T{{"1", 1}, {"2", 2}, {"3", 3}, {"4", 4}},
+		AnonInTestStruc: a,
+		NotAnon:         a,
+	}
+
+	if bench {
+		ts.Ui64 = math.MaxInt64 * 2 / 3
+		ts.Simplef.Ui64 = ts.Ui64
+	}
+
+	//For benchmarks, some things will not work.
+	if !bench {
+		//json and bson require string keys in maps
+		//ts.M = map[interface{}]interface{}{
+		//	true: "true",
+		//	int8(9): false,
+		//}
+		//gob cannot encode nil in element in array (encodeArray: nil element)
+		ts.Iptrslice = []*int64{nil, &i64a, nil, &i64b, nil, &i64c, nil, &i64d, nil}
+		// ts.Iptrslice = nil
+	}
+	if !useStringKeyOnly {
+		var _ byte = 0 // so this empty branch doesn't flag a warning
+		// ts.AnonInTestStruc.AMU32F64 = map[uint32]float64{1: 1, 2: 2, 3: 3} // Json/Bson barf
+	}
+}
+
+func newTestStruc(depth, n int, bench, useInterface, useStringKeyOnly bool) (ts *TestStruc) {
+	ts = &TestStruc{}
+	populateTestStrucCommon(&ts.TestStrucCommon, n, bench, useInterface, useStringKeyOnly)
+	if depth > 0 {
+		depth--
+		if ts.Mtsptr == nil {
+			ts.Mtsptr = make(map[string]*TestStruc)
+		}
+		if ts.Mts == nil {
+			ts.Mts = make(map[string]TestStruc)
+		}
+		ts.Mtsptr[strRpt(n, "0")] = newTestStruc(depth, n, bench, useInterface, useStringKeyOnly)
+		ts.Mts[strRpt(n, "0")] = *(ts.Mtsptr[strRpt(n, "0")])
+		ts.Its = append(ts.Its, ts.Mtsptr[strRpt(n, "0")])
+	}
+	return
+}
+
+var testStrRptMap = make(map[int]map[string]string)
+
+func strRpt(n int, s string) string {
+	if false {
+		// fmt.Printf(">>>> calling strings.Repeat on n: %d, key: %s\n", n, s)
+		return strings.Repeat(s, n)
+	}
+	m1, ok := testStrRptMap[n]
+	if !ok {
+		// fmt.Printf(">>>> making new map for n: %v\n", n)
+		m1 = make(map[string]string)
+		testStrRptMap[n] = m1
+	}
+	v1, ok := m1[s]
+	if !ok {
+		// fmt.Printf(">>>> creating new entry for key: %s\n", s)
+		v1 = strings.Repeat(s, n)
+		m1[s] = v1
+	}
+	return v1
+}
+
+// func wstrRpt(n int, s string) wrapBytes {
+// 	 return wrapBytes(bytes.Repeat([]byte(s), n))
+// }

codec/bench/x_bench_gen_test.go

@@ -0,0 +1,125 @@
+// +build x
+// +build generated
+
+// Copyright (c) 2012-2018 Ugorji Nwoke. All rights reserved.
+// Use of this source code is governed by a MIT license found in the LICENSE file.
+
+package codec
+
+import (
+	"bytes"
+	"errors"
+	"fmt"
+	"testing"
+
+	"github.com/mailru/easyjson"
+	"github.com/pquerna/ffjson/ffjson"
+	"github.com/tinylib/msgp/msgp"
+)
+
+/*
+ To update all these, use:
+ go get -u github.com/tinylib/msgp/msgp github.com/tinylib/msgp \
+           github.com/pquerna/ffjson/ffjson github.com/pquerna/ffjson \
+           github.com/mailru/easyjson/...
+
+ Known Issues with external libraries:
+ - msgp io.R/W support doesn't work. It throws error
+
+*/
+
+func init() {
+	testPreInitFns = append(testPreInitFns, benchXGenPreInit)
+}
+
+func benchXGenPreInit() {
+	benchCheckers = append(benchCheckers,
+		benchChecker{"msgp", fnMsgpEncodeFn, fnMsgpDecodeFn},
+		benchChecker{"easyjson", fnEasyjsonEncodeFn, fnEasyjsonDecodeFn},
+		benchChecker{"ffjson", fnFfjsonEncodeFn, fnFfjsonDecodeFn},
+	)
+}
+
+func fnEasyjsonEncodeFn(ts interface{}, bsIn []byte) ([]byte, error) {
+	ts2, ok := ts.(easyjson.Marshaler)
+	if !ok {
+		return nil, errors.New("easyjson: input is not a easyjson.Marshaler")
+	}
+	if testUseIoEncDec >= 0 {
+		buf := bytes.NewBuffer(bsIn[:0]) // new(bytes.Buffer)
+		_, err := easyjson.MarshalToWriter(ts2, buf)
+		return buf.Bytes(), err
+	}
+	return easyjson.Marshal(ts2)
+	// return ts.(json.Marshaler).MarshalJSON()
+}
+
+func fnEasyjsonDecodeFn(buf []byte, ts interface{}) error {
+	ts2, ok := ts.(easyjson.Unmarshaler)
+	if !ok {
+		return errors.New("easyjson: input is not a easyjson.Unmarshaler")
+	}
+	if testUseIoEncDec >= 0 {
+		return easyjson.UnmarshalFromReader(bytes.NewReader(buf), ts2)
+	}
+	return easyjson.Unmarshal(buf, ts2)
+	// return ts.(json.Unmarshaler).UnmarshalJSON(buf)
+}
+
+func fnFfjsonEncodeFn(ts interface{}, bsIn []byte) ([]byte, error) {
+	return ffjson.Marshal(ts)
+	// return ts.(json.Marshaler).MarshalJSON()
+}
+
+func fnFfjsonDecodeFn(buf []byte, ts interface{}) error {
+	return ffjson.Unmarshal(buf, ts)
+	// return ts.(json.Unmarshaler).UnmarshalJSON(buf)
+}
+
+func fnMsgpEncodeFn(ts interface{}, bsIn []byte) ([]byte, error) {
+	if _, ok := ts.(msgp.Encodable); !ok {
+		return nil, fmt.Errorf("msgp: input of type %T is not a msgp.Encodable", ts)
+	}
+	if testUseIoEncDec >= 0 {
+		buf := fnBenchmarkByteBuf(bsIn)
+		err := ts.(msgp.Encodable).EncodeMsg(msgp.NewWriter(buf))
+		return buf.Bytes(), err
+	}
+	return ts.(msgp.Marshaler).MarshalMsg(bsIn[:0]) // msgp appends to slice.
+}
+
+func fnMsgpDecodeFn(buf []byte, ts interface{}) (err error) {
+	if _, ok := ts.(msgp.Decodable); !ok {
+		return fmt.Errorf("msgp: input of type %T is not a msgp.Decodable", ts)
+	}
+	if testUseIoEncDec >= 0 {
+		err = ts.(msgp.Decodable).DecodeMsg(msgp.NewReader(bytes.NewReader(buf)))
+		return
+	}
+	_, err = ts.(msgp.Unmarshaler).UnmarshalMsg(buf)
+	return
+}
+
+func Benchmark__Msgp_______Encode(b *testing.B) {
+	fnBenchmarkEncode(b, "msgp", benchTs, fnMsgpEncodeFn)
+}
+
+func Benchmark__Msgp_______Decode(b *testing.B) {
+	fnBenchmarkDecode(b, "msgp", benchTs, fnMsgpEncodeFn, fnMsgpDecodeFn, fnBenchNewTs)
+}
+
+func Benchmark__Easyjson___Encode(b *testing.B) {
+	fnBenchmarkEncode(b, "easyjson", benchTs, fnEasyjsonEncodeFn)
+}
+
+func Benchmark__Easyjson___Decode(b *testing.B) {
+	fnBenchmarkDecode(b, "easyjson", benchTs, fnEasyjsonEncodeFn, fnEasyjsonDecodeFn, fnBenchNewTs)
+}
+
+func Benchmark__Ffjson_____Encode(b *testing.B) {
+	fnBenchmarkEncode(b, "ffjson", benchTs, fnFfjsonEncodeFn)
+}
+
+func Benchmark__Ffjson_____Decode(b *testing.B) {
+	fnBenchmarkDecode(b, "ffjson", benchTs, fnFfjsonEncodeFn, fnFfjsonDecodeFn, fnBenchNewTs)
+}

codec/bench/x_bench_test.go

@@ -0,0 +1,170 @@
+// +build x
+
+// Copyright (c) 2012-2018 Ugorji Nwoke. All rights reserved.
+// Use of this source code is governed by a MIT license found in the LICENSE file.
+
+package codec
+
+import (
+	"bytes"
+	"testing"
+
+	gcbor "bitbucket.org/bodhisnarkva/cbor/go" // gcbor "code.google.com/p/cbor/go"
+	"github.com/Sereal/Sereal/Go/sereal"
+	"github.com/davecgh/go-xdr/xdr2"
+	"github.com/json-iterator/go"
+	"gopkg.in/mgo.v2/bson"                     //"labix.org/v2/mgo/bson"
+	vmsgpack "gopkg.in/vmihailenco/msgpack.v2" //"github.com/vmihailenco/msgpack"
+)
+
+/*
+ To update all these, use:
+ go get -u github.com/tinylib/msgp/msgp github.com/tinylib/msgp \
+           github.com/pquerna/ffjson/ffjson github.com/pquerna/ffjson \
+           github.com/Sereal/Sereal/Go/sereal \
+           bitbucket.org/bodhisnarkva/cbor/go \
+           github.com/davecgh/go-xdr/xdr2 \
+           gopkg.in/mgo.v2/bson \
+           gopkg.in/vmihailenco/msgpack.v2 \
+           github.com/json-iterator/go \
+           github.com/mailru/easyjson/...
+
+ Known Issues with external libraries:
+ - msgp io.R/W support doesn't work. It throws error
+
+*/
+
+func init() {
+	testPreInitFns = append(testPreInitFns, benchXPreInit)
+}
+
+func benchXPreInit() {
+	benchCheckers = append(benchCheckers,
+		benchChecker{"json-iter", fnJsonIterEncodeFn, fnJsonIterDecodeFn},
+		benchChecker{"v-msgpack", fnVMsgpackEncodeFn, fnVMsgpackDecodeFn},
+		benchChecker{"bson", fnBsonEncodeFn, fnBsonDecodeFn},
+		// place codecs with issues at the end, so as not to make results too ugly
+		benchChecker{"gcbor", fnGcborEncodeFn, fnGcborDecodeFn}, // this logs fat ugly message, but we log.SetOutput(ioutil.Discard)
+		benchChecker{"xdr", fnXdrEncodeFn, fnXdrDecodeFn},
+		benchChecker{"sereal", fnSerealEncodeFn, fnSerealDecodeFn},
+	)
+}
+
+func fnVMsgpackEncodeFn(ts interface{}, bsIn []byte) ([]byte, error) {
+	if testUseIoEncDec >= 0 {
+		buf := bytes.NewBuffer(bsIn[:0]) // new(bytes.Buffer)
+		err := vmsgpack.NewEncoder(buf).Encode(ts)
+		return buf.Bytes(), err
+	}
+	return vmsgpack.Marshal(ts)
+}
+
+func fnVMsgpackDecodeFn(buf []byte, ts interface{}) error {
+	if testUseIoEncDec >= 0 {
+		return vmsgpack.NewDecoder(bytes.NewReader(buf)).Decode(ts)
+	}
+	return vmsgpack.Unmarshal(buf, ts)
+}
+
+func fnBsonEncodeFn(ts interface{}, bsIn []byte) ([]byte, error) {
+	return bson.Marshal(ts)
+}
+
+func fnBsonDecodeFn(buf []byte, ts interface{}) error {
+	return bson.Unmarshal(buf, ts)
+}
+
+func fnJsonIterEncodeFn(ts interface{}, bsIn []byte) ([]byte, error) {
+	if testUseIoEncDec >= 0 {
+		buf := bytes.NewBuffer(bsIn[:0]) // new(bytes.Buffer)
+		err := jsoniter.NewEncoder(buf).Encode(ts)
+		return buf.Bytes(), err
+	}
+	return jsoniter.Marshal(ts)
+}
+
+func fnJsonIterDecodeFn(buf []byte, ts interface{}) error {
+	if testUseIoEncDec >= 0 {
+		return jsoniter.NewDecoder(bytes.NewReader(buf)).Decode(ts)
+	}
+	return jsoniter.Unmarshal(buf, ts)
+}
+
+func fnXdrEncodeFn(ts interface{}, bsIn []byte) ([]byte, error) {
+	buf := fnBenchmarkByteBuf(bsIn)
+	i, err := xdr.Marshal(buf, ts)
+	return buf.Bytes()[:i], err
+}
+
+func fnXdrDecodeFn(buf []byte, ts interface{}) error {
+	_, err := xdr.Unmarshal(bytes.NewReader(buf), ts)
+	return err
+}
+
+func fnSerealEncodeFn(ts interface{}, bsIn []byte) ([]byte, error) {
+	return sereal.Marshal(ts)
+}
+
+func fnSerealDecodeFn(buf []byte, ts interface{}) error {
+	return sereal.Unmarshal(buf, ts)
+}
+
+func fnGcborEncodeFn(ts interface{}, bsIn []byte) (bs []byte, err error) {
+	buf := fnBenchmarkByteBuf(bsIn)
+	err = gcbor.NewEncoder(buf).Encode(ts)
+	return buf.Bytes(), err
+}
+
+func fnGcborDecodeFn(buf []byte, ts interface{}) error {
+	return gcbor.NewDecoder(bytes.NewReader(buf)).Decode(ts)
+}
+
+func Benchmark__JsonIter___Encode(b *testing.B) {
+	fnBenchmarkEncode(b, "jsoniter", benchTs, fnJsonIterEncodeFn)
+}
+
+func Benchmark__JsonIter___Decode(b *testing.B) {
+	fnBenchmarkDecode(b, "jsoniter", benchTs, fnJsonIterEncodeFn, fnJsonIterDecodeFn, fnBenchNewTs)
+}
+
+// Place codecs with issues at the bottom, so as not to make results look too ugly.
+
+func Benchmark__Bson_______Encode(b *testing.B) {
+	fnBenchmarkEncode(b, "bson", benchTs, fnBsonEncodeFn)
+}
+
+func Benchmark__Bson_______Decode(b *testing.B) {
+	fnBenchmarkDecode(b, "bson", benchTs, fnBsonEncodeFn, fnBsonDecodeFn, fnBenchNewTs)
+}
+
+func Benchmark__VMsgpack___Encode(b *testing.B) {
+	fnBenchmarkEncode(b, "v-msgpack", benchTs, fnVMsgpackEncodeFn)
+}
+
+func Benchmark__VMsgpack___Decode(b *testing.B) {
+	fnBenchmarkDecode(b, "v-msgpack", benchTs, fnVMsgpackEncodeFn, fnVMsgpackDecodeFn, fnBenchNewTs)
+}
+
+func Benchmark__Gcbor______Encode(b *testing.B) {
+	fnBenchmarkEncode(b, "gcbor", benchTs, fnGcborEncodeFn)
+}
+
+func Benchmark__Gcbor______Decode(b *testing.B) {
+	fnBenchmarkDecode(b, "gcbor", benchTs, fnGcborEncodeFn, fnGcborDecodeFn, fnBenchNewTs)
+}
+
+func Benchmark__Xdr________Encode(b *testing.B) {
+	fnBenchmarkEncode(b, "xdr", benchTs, fnXdrEncodeFn)
+}
+
+func Benchmark__Xdr________Decode(b *testing.B) {
+	fnBenchmarkDecode(b, "xdr", benchTs, fnXdrEncodeFn, fnXdrDecodeFn, fnBenchNewTs)
+}
+
+func Benchmark__Sereal_____Encode(b *testing.B) {
+	fnBenchmarkEncode(b, "sereal", benchTs, fnSerealEncodeFn)
+}
+
+func Benchmark__Sereal_____Decode(b *testing.B) {
+	fnBenchmarkDecode(b, "sereal", benchTs, fnSerealEncodeFn, fnSerealDecodeFn, fnBenchNewTs)
+}

codec/bench/z_all_bench_test.go

@@ -0,0 +1,255 @@
+// Copyright (c) 2012-2018 Ugorji Nwoke. All rights reserved.
+// Use of this source code is governed by a MIT license found in the LICENSE file.
+
+// +build alltests codecgen
+// +build go1.7
+
+package codec
+
+// see notes in z_all_test.go
+
+import (
+	"strconv"
+	"sync"
+	"testing"
+	"time"
+)
+
+import . "github.com/ugorji/go/codec"
+
+var benchmarkGroupOnce sync.Once
+
+var benchmarkGroupSave struct {
+	testUseIoEncDec int
+	testUseReset    bool
+	testInternStr   bool
+
+	benchDepth            int
+	benchMapStringKeyOnly bool
+	benchInitDebug        bool
+	benchVerify           bool
+	benchDoInitBench      bool
+	benchUnscientificRes  bool
+}
+
+func benchmarkGroupInitAll() {
+	testInitAll() // calls flag.Parse
+	benchmarkGroupSave.testUseIoEncDec = testUseIoEncDec
+	benchmarkGroupSave.testUseReset = testUseReset
+	benchmarkGroupSave.testInternStr = testInternStr
+
+	benchmarkGroupSave.benchDepth = benchDepth
+	benchmarkGroupSave.benchMapStringKeyOnly = benchMapStringKeyOnly
+	benchmarkGroupSave.benchInitDebug = benchInitDebug
+	benchmarkGroupSave.benchVerify = benchVerify
+	benchmarkGroupSave.benchDoInitBench = benchDoInitBench
+	benchmarkGroupSave.benchUnscientificRes = benchUnscientificRes
+}
+
+func benchmarkGroupReset() {
+	testUseIoEncDec = benchmarkGroupSave.testUseIoEncDec
+	testUseReset = benchmarkGroupSave.testUseReset
+	testInternStr = benchmarkGroupSave.testInternStr
+
+	benchDepth = benchmarkGroupSave.benchDepth
+	benchMapStringKeyOnly = benchmarkGroupSave.benchMapStringKeyOnly
+	benchInitDebug = benchmarkGroupSave.benchInitDebug
+	benchVerify = benchmarkGroupSave.benchVerify
+	benchDoInitBench = benchmarkGroupSave.benchDoInitBench
+	benchUnscientificRes = benchmarkGroupSave.benchUnscientificRes
+}
+
+func benchmarkDivider() {
+	// logT(nil, "-------------------------------\n")
+	println()
+}
+
+func benchmarkOneFn(fns []func(*testing.B)) func(*testing.B) {
+	switch len(fns) {
+	case 0:
+		return nil
+	case 1:
+		return fns[0]
+	default:
+		return func(t *testing.B) {
+			for _, f := range fns {
+				f(t)
+			}
+		}
+	}
+}
+
+func benchmarkSuiteNoop(b *testing.B) {
+	testOnce.Do(testInitAll)
+	b.ResetTimer()
+	for i := 0; i < b.N; i++ {
+		time.Sleep(10 * time.Millisecond)
+	}
+}
+
+func benchmarkSuite(t *testing.B, fns ...func(t *testing.B)) {
+	benchmarkGroupOnce.Do(benchmarkGroupInitAll)
+
+	f := benchmarkOneFn(fns)
+	// find . -name "*_test.go" | xargs grep -e 'flag.' | cut -d '&' -f 2 | cut -d ',' -f 1 | grep -e '^bench'
+
+	testReinit() // so flag.Parse() is called first, and never called again
+	benchReinit()
+
+	testDecodeOptions = DecodeOptions{}
+	testEncodeOptions = EncodeOptions{}
+
+	benchmarkGroupReset()
+
+	benchVerify = true
+	benchDoInitBench = true
+	benchUnscientificRes = true
+	testReinit()
+	benchReinit()
+	t.Run("init-metrics....", func(t *testing.B) { t.Run("Benchmark__Noop.............", benchmarkSuiteNoop) })
+
+	benchmarkGroupReset()
+
+	benchVerify = false
+	benchDoInitBench = false
+	benchUnscientificRes = false
+
+	testReinit()
+	benchReinit()
+	t.Run("options-false...", f)
+
+	benchmarkGroupReset()
+
+	testUseIoEncDec = 1024
+	testReinit()
+	benchReinit()
+	t.Run("use-bufio-!bytes", f)
+
+	benchmarkGroupReset()
+
+	testUseReset = true
+	testReinit()
+	benchReinit()
+	t.Run("reset-enc-dec...", f)
+
+	benchmarkGroupReset()
+
+	// intern string only applies to binc: don't do a full run of it
+	// testInternStr = true
+	// testReinit()
+	// benchReinit()
+	// t.Run("intern-strings", f)
+	// testInternStr = false
+
+	// benchVerify is kinda lame - serves no real purpose.
+	// benchVerify = true
+	// testReinit()
+	// benchReinit()
+	// t.Run("verify-on-decode", f)
+	// benchVerify = false
+}
+
+func benchmarkQuickSuite(t *testing.B, name string, fns ...func(t *testing.B)) {
+	benchmarkDivider()
+	benchmarkGroupOnce.Do(benchmarkGroupInitAll)
+	f := benchmarkOneFn(fns)
+
+	benchmarkGroupReset()
+
+	// bd=1 2 | ti=-1, 1024 |
+
+	testUseIoEncDec = -1
+	// benchDepth = depth
+	testReinit()
+	benchReinit()
+
+	t.Run(name+"-bd"+strconv.Itoa(benchDepth)+"........", f)
+
+	// encoded size of TestStruc is between 20K and 30K for bd=1 // consider buffer=1024 * 16 * benchDepth
+	testUseIoEncDec = 1024 // (value of defEncByteBufSize): use smaller buffer, and more flushes - it's ok.
+	// benchDepth = depth
+	testReinit()
+	benchReinit()
+	t.Run(name+"-bd"+strconv.Itoa(benchDepth)+"-buf"+strconv.Itoa(testUseIoEncDec), f)
+
+	testUseIoEncDec = 0
+	// benchDepth = depth
+	testReinit()
+	benchReinit()
+	t.Run(name+"-bd"+strconv.Itoa(benchDepth)+"-io.....", f)
+
+	benchmarkGroupReset()
+}
+
+/*
+z='bench_test.go'
+find . -name "$z" | xargs grep -e '^func Benchmark.*Encode' | \
+    cut -d '(' -f 1 | cut -d ' ' -f 2 | \
+    while read f; do echo "t.Run(\"$f\", $f)"; done &&
+echo &&
+find . -name "$z" | xargs grep -e '^func Benchmark.*Decode' | \
+    cut -d '(' -f 1 | cut -d ' ' -f 2 | \
+    while read f; do echo "t.Run(\"$f\", $f)"; done
+*/
+
+func benchmarkCodecGroup(t *testing.B) {
+	benchmarkDivider()
+	t.Run("Benchmark__Msgpack____Encode", Benchmark__Msgpack____Encode)
+	t.Run("Benchmark__Binc_______Encode", Benchmark__Binc_______Encode)
+	t.Run("Benchmark__Simple_____Encode", Benchmark__Simple_____Encode)
+	t.Run("Benchmark__Cbor_______Encode", Benchmark__Cbor_______Encode)
+	t.Run("Benchmark__Json_______Encode", Benchmark__Json_______Encode)
+	t.Run("Benchmark__Std_Json___Encode", Benchmark__Std_Json___Encode)
+	t.Run("Benchmark__Gob________Encode", Benchmark__Gob________Encode)
+	// t.Run("Benchmark__Std_Xml____Encode", Benchmark__Std_Xml____Encode)
+	benchmarkDivider()
+	t.Run("Benchmark__Msgpack____Decode", Benchmark__Msgpack____Decode)
+	t.Run("Benchmark__Binc_______Decode", Benchmark__Binc_______Decode)
+	t.Run("Benchmark__Simple_____Decode", Benchmark__Simple_____Decode)
+	t.Run("Benchmark__Cbor_______Decode", Benchmark__Cbor_______Decode)
+	t.Run("Benchmark__Json_______Decode", Benchmark__Json_______Decode)
+	t.Run("Benchmark__Std_Json___Decode", Benchmark__Std_Json___Decode)
+	t.Run("Benchmark__Gob________Decode", Benchmark__Gob________Decode)
+	// t.Run("Benchmark__Std_Xml____Decode", Benchmark__Std_Xml____Decode)
+}
+
+func BenchmarkCodecSuite(t *testing.B) { benchmarkSuite(t, benchmarkCodecGroup) }
+
+func benchmarkJsonEncodeGroup(t *testing.B) {
+	t.Run("Benchmark__Json_______Encode", Benchmark__Json_______Encode)
+}
+
+func benchmarkJsonDecodeGroup(t *testing.B) {
+	t.Run("Benchmark__Json_______Decode", Benchmark__Json_______Decode)
+}
+
+func benchmarkCborEncodeGroup(t *testing.B) {
+	t.Run("Benchmark__Cbor_______Encode", Benchmark__Cbor_______Encode)
+}
+
+func benchmarkCborDecodeGroup(t *testing.B) {
+	t.Run("Benchmark__Cbor_______Decode", Benchmark__Cbor_______Decode)
+}
+
+func BenchmarkCodecQuickSuite(t *testing.B) {
+	benchmarkQuickSuite(t, "cbor", benchmarkCborEncodeGroup)
+	benchmarkQuickSuite(t, "cbor", benchmarkCborDecodeGroup)
+	benchmarkQuickSuite(t, "json", benchmarkJsonEncodeGroup)
+	benchmarkQuickSuite(t, "json", benchmarkJsonDecodeGroup)
+
+	// depths := [...]int{1, 4}
+	// for _, d := range depths {
+	// 	benchmarkQuickSuite(t, d, benchmarkJsonEncodeGroup)
+	// 	benchmarkQuickSuite(t, d, benchmarkJsonDecodeGroup)
+	// }
+
+	// benchmarkQuickSuite(t, 1, benchmarkJsonEncodeGroup)
+	// benchmarkQuickSuite(t, 4, benchmarkJsonEncodeGroup)
+	// benchmarkQuickSuite(t, 1, benchmarkJsonDecodeGroup)
+	// benchmarkQuickSuite(t, 4, benchmarkJsonDecodeGroup)
+
+	// benchmarkQuickSuite(t, 1, benchmarkJsonEncodeGroup, benchmarkJsonDecodeGroup)
+	// benchmarkQuickSuite(t, 4, benchmarkJsonEncodeGroup, benchmarkJsonDecodeGroup)
+	// benchmarkQuickSuite(t, benchmarkJsonEncodeGroup)
+	// benchmarkQuickSuite(t, benchmarkJsonDecodeGroup)
+}

codec/bench/z_all_x_bench_gen_test.go

@@ -0,0 +1,53 @@
+// Copyright (c) 2012-2018 Ugorji Nwoke. All rights reserved.
+// Use of this source code is governed by a MIT license found in the LICENSE file.
+
+// +build alltests
+// +build x
+// +build go1.7
+// +build generated
+
+package codec
+
+// see notes in z_all_bench_test.go
+
+import "testing"
+
+func benchmarkCodecXGenGroup(t *testing.B) {
+	benchmarkDivider()
+	t.Run("Benchmark__Msgpack____Encode", Benchmark__Msgpack____Encode)
+	t.Run("Benchmark__Binc_______Encode", Benchmark__Binc_______Encode)
+	t.Run("Benchmark__Simple_____Encode", Benchmark__Simple_____Encode)
+	t.Run("Benchmark__Cbor_______Encode", Benchmark__Cbor_______Encode)
+	t.Run("Benchmark__Json_______Encode", Benchmark__Json_______Encode)
+	t.Run("Benchmark__Std_Json___Encode", Benchmark__Std_Json___Encode)
+	t.Run("Benchmark__Gob________Encode", Benchmark__Gob________Encode)
+	t.Run("Benchmark__JsonIter___Encode", Benchmark__JsonIter___Encode)
+	t.Run("Benchmark__Bson_______Encode", Benchmark__Bson_______Encode)
+	t.Run("Benchmark__VMsgpack___Encode", Benchmark__VMsgpack___Encode)
+	t.Run("Benchmark__Msgp_______Encode", Benchmark__Msgp_______Encode)
+	t.Run("Benchmark__Easyjson___Encode", Benchmark__Easyjson___Encode)
+	t.Run("Benchmark__Ffjson_____Encode", Benchmark__Ffjson_____Encode)
+	// t.Run("Benchmark__Gcbor______Encode", Benchmark__Gcbor______Encode)
+	// t.Run("Benchmark__Xdr________Encode", Benchmark__Xdr________Encode)
+	t.Run("Benchmark__Sereal_____Encode", Benchmark__Sereal_____Encode)
+
+	benchmarkDivider()
+	t.Run("Benchmark__Msgpack____Decode", Benchmark__Msgpack____Decode)
+	t.Run("Benchmark__Binc_______Decode", Benchmark__Binc_______Decode)
+	t.Run("Benchmark__Simple_____Decode", Benchmark__Simple_____Decode)
+	t.Run("Benchmark__Cbor_______Decode", Benchmark__Cbor_______Decode)
+	t.Run("Benchmark__Json_______Decode", Benchmark__Json_______Decode)
+	t.Run("Benchmark__Std_Json___Decode", Benchmark__Std_Json___Decode)
+	t.Run("Benchmark__Gob________Decode", Benchmark__Gob________Decode)
+	t.Run("Benchmark__JsonIter___Decode", Benchmark__JsonIter___Decode)
+	t.Run("Benchmark__Bson_______Decode", Benchmark__Bson_______Decode)
+	// t.Run("Benchmark__VMsgpack___Decode", Benchmark__VMsgpack___Decode)
+	t.Run("Benchmark__Msgp_______Decode", Benchmark__Msgp_______Decode)
+	t.Run("Benchmark__Easyjson___Decode", Benchmark__Easyjson___Decode)
+	t.Run("Benchmark__Ffjson_____Decode", Benchmark__Ffjson_____Decode)
+	// t.Run("Benchmark__Gcbor______Decode", Benchmark__Gcbor______Decode)
+	// t.Run("Benchmark__Xdr________Decode", Benchmark__Xdr________Decode)
+	// t.Run("Benchmark__Sereal_____Decode", Benchmark__Sereal_____Decode)
+}
+
+func BenchmarkCodecXGenSuite(t *testing.B) { benchmarkSuite(t, benchmarkCodecXGenGroup) }

codec/bench/z_all_x_bench_test.go

@@ -0,0 +1,118 @@
+// Copyright (c) 2012-2018 Ugorji Nwoke. All rights reserved.
+// Use of this source code is governed by a MIT license found in the LICENSE file.
+
+// +build alltests
+// +build x
+// +build go1.7
+
+package codec
+
+// see notes in z_all_bench_test.go
+
+import "testing"
+
+// Note: The following cannot parse TestStruc effectively,
+// even with changes to remove arrays and minimize integer size to fit into int64 space.
+//
+// So we exclude them, listed below:
+// encode: gcbor, xdr
+// decode: gcbor, vmsgpack, xdr, sereal
+
+func benchmarkXGroup(t *testing.B) {
+	benchmarkDivider()
+	t.Run("Benchmark__JsonIter___Encode", Benchmark__JsonIter___Encode)
+	t.Run("Benchmark__Bson_______Encode", Benchmark__Bson_______Encode)
+	t.Run("Benchmark__VMsgpack___Encode", Benchmark__VMsgpack___Encode)
+	// t.Run("Benchmark__Gcbor______Encode", Benchmark__Gcbor______Encode)
+	// t.Run("Benchmark__Xdr________Encode", Benchmark__Xdr________Encode)
+	t.Run("Benchmark__Sereal_____Encode", Benchmark__Sereal_____Encode)
+
+	benchmarkDivider()
+	t.Run("Benchmark__JsonIter___Decode", Benchmark__JsonIter___Decode)
+	t.Run("Benchmark__Bson_______Decode", Benchmark__Bson_______Decode)
+	// t.Run("Benchmark__VMsgpack___Decode", Benchmark__VMsgpack___Decode)
+	// t.Run("Benchmark__Gcbor______Decode", Benchmark__Gcbor______Decode)
+	// t.Run("Benchmark__Xdr________Decode", Benchmark__Xdr________Decode)
+	// t.Run("Benchmark__Sereal_____Decode", Benchmark__Sereal_____Decode)
+}
+
+func benchmarkCodecXGroup(t *testing.B) {
+	benchmarkDivider()
+	t.Run("Benchmark__Msgpack____Encode", Benchmark__Msgpack____Encode)
+	t.Run("Benchmark__Binc_______Encode", Benchmark__Binc_______Encode)
+	t.Run("Benchmark__Simple_____Encode", Benchmark__Simple_____Encode)
+	t.Run("Benchmark__Cbor_______Encode", Benchmark__Cbor_______Encode)
+	t.Run("Benchmark__Json_______Encode", Benchmark__Json_______Encode)
+	t.Run("Benchmark__Std_Json___Encode", Benchmark__Std_Json___Encode)
+	t.Run("Benchmark__Gob________Encode", Benchmark__Gob________Encode)
+	// t.Run("Benchmark__Std_Xml____Encode", Benchmark__Std_Xml____Encode)
+	t.Run("Benchmark__JsonIter___Encode", Benchmark__JsonIter___Encode)
+	t.Run("Benchmark__Bson_______Encode", Benchmark__Bson_______Encode)
+	t.Run("Benchmark__VMsgpack___Encode", Benchmark__VMsgpack___Encode)
+	// t.Run("Benchmark__Gcbor______Encode", Benchmark__Gcbor______Encode)
+	// t.Run("Benchmark__Xdr________Encode", Benchmark__Xdr________Encode)
+	t.Run("Benchmark__Sereal_____Encode", Benchmark__Sereal_____Encode)
+
+	benchmarkDivider()
+	t.Run("Benchmark__Msgpack____Decode", Benchmark__Msgpack____Decode)
+	t.Run("Benchmark__Binc_______Decode", Benchmark__Binc_______Decode)
+	t.Run("Benchmark__Simple_____Decode", Benchmark__Simple_____Decode)
+	t.Run("Benchmark__Cbor_______Decode", Benchmark__Cbor_______Decode)
+	t.Run("Benchmark__Json_______Decode", Benchmark__Json_______Decode)
+	t.Run("Benchmark__Std_Json___Decode", Benchmark__Std_Json___Decode)
+	t.Run("Benchmark__Gob________Decode", Benchmark__Gob________Decode)
+	// t.Run("Benchmark__Std_Xml____Decode", Benchmark__Std_Xml____Decode)
+	t.Run("Benchmark__JsonIter___Decode", Benchmark__JsonIter___Decode)
+	t.Run("Benchmark__Bson_______Decode", Benchmark__Bson_______Decode)
+	// t.Run("Benchmark__VMsgpack___Decode", Benchmark__VMsgpack___Decode)
+	// t.Run("Benchmark__Gcbor______Decode", Benchmark__Gcbor______Decode)
+	// t.Run("Benchmark__Xdr________Decode", Benchmark__Xdr________Decode)
+	// t.Run("Benchmark__Sereal_____Decode", Benchmark__Sereal_____Decode)
+}
+
+var benchmarkXSkipMsg = `>>>> Skipping - these cannot (en|de)code TestStruc - encode (gcbor, xdr, xml), decode (gcbor, vmsgpack, xdr, sereal, xml)`
+
+func BenchmarkXSuite(t *testing.B) {
+	println(benchmarkXSkipMsg)
+	benchmarkSuite(t, benchmarkXGroup)
+}
+
+func BenchmarkCodecXSuite(t *testing.B) {
+	println(benchmarkXSkipMsg)
+	benchmarkSuite(t, benchmarkCodecXGroup)
+}
+
+func benchmarkAllJsonEncodeGroup(t *testing.B) {
+	benchmarkDivider()
+	t.Run("Benchmark__Json_______Encode", Benchmark__Json_______Encode)
+	t.Run("Benchmark__Std_Json___Encode", Benchmark__Std_Json___Encode)
+	t.Run("Benchmark__JsonIter___Encode", Benchmark__JsonIter___Encode)
+}
+
+func benchmarkAllJsonDecodeGroup(t *testing.B) {
+	benchmarkDivider()
+	t.Run("Benchmark__Json_______Decode", Benchmark__Json_______Decode)
+	t.Run("Benchmark__Std_Json___Decode", Benchmark__Std_Json___Decode)
+	t.Run("Benchmark__JsonIter___Decode", Benchmark__JsonIter___Decode)
+}
+
+func BenchmarkCodecQuickAllJsonSuite(t *testing.B) {
+	benchmarkQuickSuite(t, "json-all", benchmarkAllJsonEncodeGroup)
+	benchmarkQuickSuite(t, "json-all", benchmarkAllJsonDecodeGroup)
+
+	// depths := [...]int{1, 4}
+	// for _, d := range depths {
+	// 	benchmarkQuickSuite(t, d, benchmarkAllJsonEncodeGroup)
+	// 	benchmarkQuickSuite(t, d, benchmarkAllJsonDecodeGroup)
+	// }
+
+	// benchmarkQuickSuite(t, 1, benchmarkAllJsonEncodeGroup)
+	// benchmarkQuickSuite(t, 4, benchmarkAllJsonEncodeGroup)
+	// benchmarkQuickSuite(t, 1, benchmarkAllJsonDecodeGroup)
+	// benchmarkQuickSuite(t, 4, benchmarkAllJsonDecodeGroup)
+
+	// benchmarkQuickSuite(t, 1, benchmarkAllJsonEncodeGroup, benchmarkAllJsonDecodeGroup)
+	// benchmarkQuickSuite(t, 4, benchmarkAllJsonEncodeGroup, benchmarkAllJsonDecodeGroup)
+	// benchmarkQuickSuite(t, benchmarkAllJsonEncodeGroup)
+	// benchmarkQuickSuite(t, benchmarkAllJsonDecodeGroup)
+}

codec/build.sh

@@ -119,6 +119,9 @@ run("mammoth2-test.go.tmpl", "mammoth2_generated_test.go")
 }
 EOF
 
+    sed -e 's+// __DO_NOT_REMOVE__NEEDED_FOR_REPLACING__IMPORT_PATH__FOR_CODEC_BENCH__+import . "github.com/ugorji/go/codec"+' \
+        shared_test.go > bench/shared_test.go
+    
     # explicitly return 0 if this passes, else return 1
     go run -tags "notfastpath safe codecgen.exec" gen-from-tmpl.generated.go &&
         rm -f gen-from-tmpl.*generated.go &&
@@ -127,43 +130,45 @@ EOF
 }
 
 _codegenerators() {
-    if ! [[ $zforce || $(_ng "values_codecgen${zsfx}") ]]; then return 0; fi
+    local c5="_generated_test.go"
+    local c7="$PWD/codecgen"
+    local c8="$c7/__codecgen"
+    local c9="codecgen-scratch.go"
 
+    if ! [[ $zforce || $(_ng "values_codecgen${c5}") ]]; then return 0; fi
+    
     # Note: ensure you run the codecgen for this codebase/directory i.e. ./codecgen/codecgen
-    local c9="codecgen-scratch.go"
-    local c7="$zmydir/codecgen"
-    local c8="$c7/__codecgen"
     true &&
         echo "codecgen ... " &&
         if [[ $zforce || ! -f "$c8" || "$c7/gen.go" -nt "$c8" ]]; then
             echo "rebuilding codecgen ... " && ( cd codecgen && go build -o $c8 ${zargs[*]} . )
         fi &&
-        $c8 -rt codecgen -t 'codecgen generated' -o values_codecgen${zsfx} -d 19780 $zfin $zfin2 &&
+        $c8 -rt codecgen -t 'codecgen generated' -o values_codecgen${c5} -d 19780 $zfin $zfin2 &&
         cp mammoth2_generated_test.go $c9 &&
-        $c8 -t '!notfastpath' -o mammoth2_codecgen${zsfx} -d 19781 mammoth2_generated_test.go &&
+        $c8 -t '!notfastpath' -o mammoth2_codecgen${c5} -d 19781 mammoth2_generated_test.go &&
         rm -f $c9 &&
         echo "generators done!" 
 }
 
 _prebuild() {
     echo "prebuild: zforce: $zforce"
-    zmydir=`pwd`
+    local d="$PWD"
     zfin="test_values.generated.go"
     zfin2="test_values_flex.generated.go"
-    zsfx="_generated_test.go"
     zpkg="github.com/ugorji/go/codec"
-    # zpkg=${zmydir##*/src/}
-    # zgobase=${zmydir%%/src/*}
+    # zpkg=${d##*/src/}
+    # zgobase=${d%%/src/*}
     # rm -f *_generated_test.go 
     rm -f codecgen-*.go &&
         _build &&
-        cp $zmydir/values_test.go $zmydir/$zfin &&
-        cp $zmydir/values_flex_test.go $zmydir/$zfin2 &&
+        cp $d/values_test.go $d/$zfin &&
+        cp $d/values_flex_test.go $d/$zfin2 &&
         _codegenerators &&
         if [[ "$(type -t _codegenerators_external )" = "function" ]]; then _codegenerators_external ; fi &&
         if [[ $zforce ]]; then go install ${zargs[*]} .; fi &&
         echo "prebuild done successfully"
-    rm -f $zmydir/$zfin $zmydir/$zfin2 
+    rm -f $d/$zfin $d/$zfin2
+    unset zfin zfin2 zpkg
 }
 
 _make() {
@@ -255,7 +260,7 @@ _main() {
         'xz') _analyze "$@" ;;
         'xb') _bench "$@" ;;
     esac
-    unset zforce
+    unset zforce zargs zbenchflags
 }
 
 [ "." = `dirname $0` ] && _main "$@"

codec/shared_test.go

@@ -51,6 +51,8 @@ import (
 	"testing"
 )
 
+// __DO_NOT_REMOVE__NEEDED_FOR_REPLACING__IMPORT_PATH__FOR_CODEC_BENCH__
+
 type testHED struct {
 	H Handle
 	E *Encoder