codec: Robust Msgpack Spec RPC, Support for Go 1.2 Binary(M/Unm)arshaler, Major refactoring.

With Go 1.2 Binary(M/Unm)arshaler support, you can have a type implement the
respective (M/Unm)arshalBinary methods and we call those to encode or decode the type.

With Robust Msgpack Spec RPC support, you can now call RPC functions in a python/ruby/c server
over msgpack spec rpc, even when the functions take multiple arguments/parameters.

Adding support for Go 1.2 Binary(M/Unm)arshaler incurred an additional
cost to encoding and decoding. Some refactoring was done based on anectodal
testing to try to get back some of the performance loss:

  - Use pointer receivers more. Dereferencing much cheaper than copy (allocation)
  - Do not continually perform checks where the result can be done once and cached.
    Instead, do the checks once and cache the function to be called, and call it directly
    next time the value is seen.
  - Since a Decoder is not safe for concurrent access, using a cache at Encoder/Decoder
    level will allow us operate lock-free.
  - Simplify codebase to use one extHandle
    (instead of duplicate encHandle and decHandle which look similar)
  - look through use of wasNilIntf bool to ensure it's only used when necessary.
    Makes implementation of cached functions better
  - Do not use closures or receiver-curried functions .
    They have a clear and significant cost. Instead, use direct functions, collect the variables
    and pass them to the function directly.

All in all, there is a performance cost to these changes, but the code now provides the one feature
that JSON/Gob/etc provided previously i.e. ability to have a type define how it encodes or decodes
itself without having to register an extension at the En/Decoder level. You can still use an
extension function to override it.

codec/0doc.go

@@ -6,7 +6,7 @@ High Performance, Feature-Rich Idiomatic Go encoding library for msgpack and bin
 
 Supported Serialization formats are:
 
-  - msgpack: [http://wiki.msgpack.org/display/MSGPACK/Format+specification]
+  - msgpack: [https://github.com/msgpack/msgpack]
   - binc: [http://github.com/ugorji/binc]
 
 To install:
@@ -19,41 +19,42 @@ the standard library (ie json, xml, gob, etc).
 Rich Feature Set includes:
 
   - Simple but extremely powerful and feature-rich API
-  - Very High Performance.
+  - Very High Performance.   
     Our extensive benchmarks show us outperforming Gob, Json and Bson by 2-4X.
     This was achieved by taking extreme care on:
       - managing allocation
-      - stack frame size (important due to Go's use of split stacks),
+      - stack frame size (important due to Go's use of split stacks), 
       - reflection use
       - recursion implications
       - zero-copy mode (encoding/decoding to byte slice without using temp buffers)
-  - Correct.
-    Care was taken to precisely handle corner cases like:
+  - Correct.  
+    Care was taken to precisely handle corner cases like: 
       overflows, nil maps and slices, nil value in stream, etc.
-  - Efficient zero-copying into temporary byte buffers
+  - Efficient zero-copying into temporary byte buffers  
     when encoding into or decoding from a byte slice.
   - Standard field renaming via tags
-  - Encoding from any value
+  - Encoding from any value  
     (struct, slice, map, primitives, pointers, interface{}, etc)
-  - Decoding into pointer to any non-nil typed value
+  - Decoding into pointer to any non-nil typed value  
     (struct, slice, map, int, float32, bool, string, reflect.Value, etc)
   - Supports extension functions to handle the encode/decode of custom types
-  - Schema-less decoding
-    (decode into a pointer to a nil interface{} as opposed to a typed non-nil value).
-    Includes Options to configure what specific map or slice type to use
+  - Support Go 1.2 encoding.BinaryMarshaler/BinaryUnmarshaler
+  - Schema-less decoding  
+    (decode into a pointer to a nil interface{} as opposed to a typed non-nil value).  
+    Includes Options to configure what specific map or slice type to use 
     when decoding an encoded list or map into a nil interface{}
   - Provides a RPC Server and Client Codec for net/rpc communication protocol.
   - Msgpack Specific:
       - Provides extension functions to handle spec-defined extensions (binary, timestamp)
-      - Options to resolve ambiguities in handling raw bytes (as string or []byte)
+      - Options to resolve ambiguities in handling raw bytes (as string or []byte)  
         during schema-less decoding (decoding into a nil interface{})
-      - RPC Server/Client Codec for msgpack-rpc protocol defined at:
+      - RPC Server/Client Codec for msgpack-rpc protocol defined at: 
         http://wiki.msgpack.org/display/MSGPACK/RPC+specification
 
 Extension Support
 
 Users can register a function to handle the encoding or decoding of
-their custom types.
+their custom types. 
 
 There are no restrictions on what the custom type can be. Extensions can
 be any type: pointers, structs, custom types off arrays/slices, strings,
@@ -84,7 +85,7 @@ Typical usage model:
       sliceByteTyp = reflect.TypeOf([]byte(nil))
       timeTyp = reflect.TypeOf(time.Time{})
     )
-
+    
     // create and configure Handle
     var (
       bh codec.BincHandle
@@ -92,7 +93,7 @@ Typical usage model:
     )
 
     mh.MapType = mapStrIntfTyp
-
+    
     // configure extensions for msgpack, to enable Binary and Time support for tags 0 and 1
     mh.AddExt(sliceByteTyp, 0, mh.BinaryEncodeExt, mh.BinaryDecodeExt)
     mh.AddExt(timeTyp, 1, mh.TimeEncodeExt, mh.TimeDecodeExt)
@@ -104,15 +105,15 @@ Typical usage model:
       b []byte
       h = &bh // or mh to use msgpack
     )
-
+    
     dec = codec.NewDecoder(r, h)
     dec = codec.NewDecoderBytes(b, h)
-    err = dec.Decode(&v)
-
+    err = dec.Decode(&v) 
+    
     enc = codec.NewEncoder(w, h)
     enc = codec.NewEncoderBytes(&b, h)
     err = enc.Encode(v)
-
+    
     //RPC Server
     go func() {
         for {
@@ -122,9 +123,9 @@ Typical usage model:
             rpc.ServeCodec(rpcCodec)
         }
     }()
-
+    
     //RPC Communication (client side)
-    conn, err = net.Dial("tcp", "localhost:5555")
+    conn, err = net.Dial("tcp", "localhost:5555")  
     rpcCodec := codec.GoRpc.ClientCodec(conn, h)
     //OR rpcCodec := codec.MsgpackSpecRpc.ClientCodec(conn, h)
     client := rpc.NewClientWithCodec(rpcCodec)
@@ -133,33 +134,37 @@ Representative Benchmark Results
 
 A sample run of benchmark using "go test -bi -bench=.":
 
-    ..............................................
-    Benchmark:
-    	Struct recursive Depth:             1
-    	ApproxDeepSize Of benchmark Struct: 4786
-    Benchmark One-Pass Run:
-    	   msgpack: len: 1564
-    	      binc: len: 1191
-    	       gob: len: 1972
-    	      json: len: 2538
-    	 v-msgpack: len: 1600
-    	      bson: len: 3025
-    ..............................................
-    PASS
-    Benchmark__Msgpack__Encode	   50000	     61731 ns/op
-    Benchmark__Msgpack__Decode	   10000	    115947 ns/op
-    Benchmark__Binc_____Encode	   50000	     64568 ns/op
-    Benchmark__Binc_____Decode	   10000	    113843 ns/op
-    Benchmark__Gob______Encode	   10000	    143956 ns/op
-    Benchmark__Gob______Decode	    5000	    431889 ns/op
-    Benchmark__Json_____Encode	   10000	    158662 ns/op
-    Benchmark__Json_____Decode	    5000	    310744 ns/op
-    Benchmark__Bson_____Encode	   10000	    172905 ns/op
-    Benchmark__Bson_____Decode	   10000	    228564 ns/op
-    Benchmark__VMsgpack_Encode	   20000	     81752 ns/op
-    Benchmark__VMsgpack_Decode	   10000	    160050 ns/op
-
-To run full benchmark suite (including against vmsgpack and bson),
+   ..............................................
+   BENCHMARK INIT: 2013-09-30 14:18:26.997930788 -0400 EDT
+   To run full benchmark comparing encodings (MsgPack, Binc, JSON, GOB, etc), use: "go test -bench=."
+   Benchmark: 
+      	Struct recursive Depth:             1
+      	ApproxDeepSize Of benchmark Struct: 4694 bytes
+   Benchmark One-Pass Run:
+      	 v-msgpack: len: 1600 bytes
+      	      bson: len: 3025 bytes
+      	   msgpack: len: 1560 bytes
+      	      binc: len: 1187 bytes
+      	       gob: len: 1972 bytes
+      	      json: len: 2538 bytes
+   ..............................................
+   PASS
+   Benchmark__Msgpack__Encode	   50000	     69408 ns/op	   15852 B/op	      84 allocs/op
+   Benchmark__Msgpack__Decode	   10000	    119152 ns/op	   15542 B/op	     424 allocs/op
+   Benchmark__Binc_____Encode	   20000	     80940 ns/op	   18033 B/op	      88 allocs/op
+   Benchmark__Binc_____Decode	   10000	    123617 ns/op	   16363 B/op	     305 allocs/op
+   Benchmark__Gob______Encode	   10000	    152634 ns/op	   21342 B/op	     238 allocs/op
+   Benchmark__Gob______Decode	    5000	    424450 ns/op	   83625 B/op	    1842 allocs/op
+   Benchmark__Json_____Encode	   20000	     83246 ns/op	   13866 B/op	     102 allocs/op
+   Benchmark__Json_____Decode	   10000	    263762 ns/op	   14166 B/op	     493 allocs/op
+   Benchmark__Bson_____Encode	   10000	    129876 ns/op	   27722 B/op	     514 allocs/op
+   Benchmark__Bson_____Decode	   10000	    164583 ns/op	   16478 B/op	     789 allocs/op
+   Benchmark__VMsgpack_Encode	   50000	     71333 ns/op	   12356 B/op	     343 allocs/op
+   Benchmark__VMsgpack_Decode	   10000	    161800 ns/op	   20302 B/op	     571 allocs/op
+   ok  	ugorji.net/codec	27.165s
+
+
+To run full benchmark suite (including against vmsgpack and bson), 
 see notes in ext_dep_test.go
 
 */

codec/binc.go

@@ -25,7 +25,7 @@ import (
 //    Unicode_Other Binc types (UTF16, UTF32) are currently unsupported.
 //Note that these EXCEPTIONS are temporary and full support is possible and may happen soon.
 type BincHandle struct {
-	encdecHandle
+	extHandle
 	EncodeOptions
 	DecodeOptions
 }
@@ -105,15 +105,17 @@ func (_ *BincHandle) writeExt() bool {
 	return true
 }
 
-func (e *bincEncDriver) encodeBuiltinType(rt uintptr, rv reflect.Value) bool {
+func (e *bincEncDriver) isBuiltinType(rt uintptr) bool {
+	return rt == timeTypId
+}
+
+func (e *bincEncDriver) encodeBuiltinType(rt uintptr, rv reflect.Value) {
 	switch rt {
 	case timeTypId:
 		bs := encodeTime(rv.Interface().(time.Time))
 		e.w.writen1(bincVdTimestamp<<4 | uint8(len(bs)))
 		e.w.writeb(bs)
-		return true
 	}
-	return false
 }
 
 func (e *bincEncDriver) encodeNil() {
@@ -402,7 +404,11 @@ func (d *bincDecDriver) tryDecodeAsNil() bool {
 	return false
 }
 
-func (d *bincDecDriver) decodeBuiltinType(rt uintptr, rv reflect.Value) bool {
+func (d *bincDecDriver) isBuiltinType(rt uintptr) bool {
+	return rt == timeTypId
+}
+
+func (d *bincDecDriver) decodeBuiltinType(rt uintptr, rv reflect.Value) {
 	switch rt {
 	case timeTypId:
 		if d.vd != bincVdTimestamp {
@@ -414,9 +420,7 @@ func (d *bincDecDriver) decodeBuiltinType(rt uintptr, rv reflect.Value) bool {
 		}
 		rv.Set(reflect.ValueOf(tt))
 		d.bdRead = false
-		return true
 	}
-	return false
 }
 
 func (d *bincDecDriver) decFloatPre(vs, defaultLen byte) {
@@ -834,6 +838,3 @@ func (d *bincDecDriver) decodeNaked() (rv reflect.Value, ctx decodeNakedContext)
 	return
 }
 
-
-
-

codec/codecs_test.go

@@ -36,6 +36,8 @@ import (
 	"strconv"
 	"testing"
 	"time"
+	"sync/atomic"
+	"runtime"
 )
 
 type testVerifyArg int
@@ -44,6 +46,7 @@ const (
 	testVerifyMapTypeSame testVerifyArg = iota
 	testVerifyMapTypeStrIntf
 	testVerifyMapTypeIntfIntf
+	// testVerifySliceIntf
 	testVerifyForPython
 )
 
@@ -53,7 +56,11 @@ var (
 	testStructToArray bool
 	_                           = fmt.Printf
 	skipVerifyVal   interface{} = &(struct{}{})
-	timeLoc                     = time.FixedZone("UTC-08:00", -8*60*60)         //time.UTC-8
+
+	// For Go Time, do not use a descriptive timezone. 
+	// It's unnecessary, and makes it harder to do a reflect.DeepEqual.
+	// The Offset already tells what the offset should be, if not on UTC and unknown zone name.
+	timeLoc                     = time.FixedZone("", -8*60*60)         // UTC-08:00 //time.UTC-8
 	timeToCompare1              = time.Date(2012, 2, 2, 2, 2, 2, 2000, timeLoc) 
 	timeToCompare2              = time.Date(1900, 2, 2, 2, 2, 2, 2000, timeLoc) 
 	timeToCompare3              = time.Unix(0, 0).UTC()
@@ -76,20 +83,6 @@ func testInitFlags() {
 	flag.BoolVar(&testStructToArray, "ts2a", false, "Set StructToArray option")
 }	
 
-func testInit() {	
-	gob.Register(new(TestStruc))
-	if testInitDebug {
-		ts0 := newTestStruc(2, false)
-		fmt.Printf("====> depth: %v, ts: %#v\n", 2, ts0)
-	}
-
-	testBincH.StructToArray = testStructToArray
-	testMsgpackH.StructToArray = testStructToArray
-	testMsgpackH.RawToString = true 
-	//testMsgpackH.AddExt(byteSliceTyp, 0, testMsgpackH.BinaryEncodeExt, testMsgpackH.BinaryDecodeExt)
-	testMsgpackH.AddExt(timeTyp, 1, testMsgpackH.TimeEncodeExt, testMsgpackH.TimeDecodeExt)
-}
-
 type AnonInTestStruc struct {
 	AS        string
 	AI64      int64
@@ -136,6 +129,10 @@ type TestStruc struct {
 	Nteststruc *TestStruc
 }
 
+type TestABC struct {
+	A, B, C string
+}
+
 type TestRpcInt struct {
 	i int
 }
@@ -143,6 +140,14 @@ type TestRpcInt struct {
 func (r *TestRpcInt) Update(n int, res *int) error      { r.i = n; *res = r.i; return nil }
 func (r *TestRpcInt) Square(ignore int, res *int) error { *res = r.i * r.i; return nil }
 func (r *TestRpcInt) Mult(n int, res *int) error        { *res = r.i * n; return nil }
+func (r *TestRpcInt) EchoStruct(arg TestABC, res *string) error { 
+	*res = fmt.Sprintf("%#v", arg)
+	return nil
+}
+func (r *TestRpcInt) Echo123(args []string, res *string) error {
+	*res = fmt.Sprintf("%#v", args)
+	return nil
+}
 
 func testVerifyVal(v interface{}, arg testVerifyArg) (v2 interface{}) {
 	//for python msgpack, 
@@ -256,7 +261,18 @@ func testVerifyVal(v interface{}, arg testVerifyArg) (v2 interface{}) {
 	return
 }
 
-func init() {
+func testInit() {	
+	gob.Register(new(TestStruc))
+	if testInitDebug {
+		ts0 := newTestStruc(2, false)
+		fmt.Printf("====> depth: %v, ts: %#v\n", 2, ts0)
+	}
+
+	testBincH.StructToArray = testStructToArray
+	testMsgpackH.StructToArray = testStructToArray
+	testMsgpackH.RawToString = true 
+	//testMsgpackH.AddExt(byteSliceTyp, 0, testMsgpackH.BinaryEncodeExt, testMsgpackH.BinaryDecodeExt)
+	testMsgpackH.AddExt(timeTyp, 1, testMsgpackH.TimeEncodeExt, testMsgpackH.TimeDecodeExt)
 	primitives := []interface{}{
 		int8(-8),
 		int16(-1616),
@@ -501,6 +517,7 @@ func doTestCodecTableOne(t *testing.T, testNil bool, h Handle,
 			continue
 		}
 
+		// debugf("=============>>>> %#v", v0check)
 		if err = deepEqual(v0check, v1); err == nil {
 			logT(t, "++++++++ Before and After marshal matched\n")
 		} else {
@@ -658,63 +675,42 @@ func testCodecMiscOne(t *testing.T, h Handle) {
 	}
 }
 
-func doTestRpcOne(t *testing.T, rr Rpc, h Handle, callClose, doRequest, doExit bool) {
+func doTestRpcOne(t *testing.T, rr Rpc, h Handle, doRequest bool, exitSleepMs time.Duration,
+) (port int) {
 	srv := rpc.NewServer()
 	srv.Register(testRpcInt)
 	ln, err := net.Listen("tcp", "127.0.0.1:0")
 	// log("listener: %v", ln.Addr())
 	checkErrT(t, err)
-	defer ln.Close()
-
+	port = (ln.Addr().(*net.TCPAddr)).Port
 	// var opts *DecoderOptions
 	// opts := testDecOpts
 	// opts.MapType = mapStringIntfTyp
 	// opts.RawToString = false
 	serverExitChan := make(chan bool, 1)
+	var serverExitFlag uint64 = 0
 	serverFn := func() {
 		for {
 			conn1, err1 := ln.Accept()
 			if err1 != nil {
+				//fmt.Printf("accept err1: %v\n", err1)
 				continue
 			}
-			bs := make([]byte, 1)
-			n1, err1 := conn1.Read(bs)
-			if n1 != 1 || err1 != nil {
-				conn1.Close()
-				continue
-			}
-			var sc rpc.ServerCodec
-			switch bs[0] {
-			case 'R':
-				sc = rr.ServerCodec(conn1, h)
-			case 'X':
+			if atomic.LoadUint64(&serverExitFlag) == 1 {
 				serverExitChan <- true
-				// <- serverExitChan
 				conn1.Close()
 				return // exit serverFn goroutine
 			}
-			if sc == nil {
-				conn1.Close()
-				continue
-			}
+			var sc rpc.ServerCodec = rr.ServerCodec(conn1, h)
 			srv.ServeCodec(sc)
-			// for {
-			// 	if err1 = srv.ServeRequest(sc); err1 != nil {
-			// 		break
-			// 	}
-			// }
-			// if callClose {
-			// 	sc.Close()
-			// }
 		}
 	}
 
 	clientFn := func(cc rpc.ClientCodec) {
 		cl := rpc.NewClientWithCodec(cc)
-		if callClose {
-			defer cl.Close()
-		}
+		defer cl.Close()
 		var up, sq, mult int
+		var rstr string
 		// log("Calling client")
 		checkErrT(t, cl.Call("TestRpcInt.Update", 5, &up))
 		// log("Called TestRpcInt.Update")
@@ -724,31 +720,48 @@ func doTestRpcOne(t *testing.T, rr Rpc, h Handle, callClose, doRequest, doExit b
 		checkEqualT(t, sq, 25)
 		checkErrT(t, cl.Call("TestRpcInt.Mult", 20, &mult))
 		checkEqualT(t, mult, 100)
+		checkErrT(t, cl.Call("TestRpcInt.EchoStruct", TestABC{"Aa", "Bb", "Cc"}, &rstr))
+		checkEqualT(t, rstr, fmt.Sprintf("%#v", TestABC{"Aa", "Bb", "Cc"}))
+		checkErrT(t, cl.Call("TestRpcInt.Echo123", []string{"A1", "B2", "C3"}, &rstr))
+		checkEqualT(t, rstr, fmt.Sprintf("%#v", []string{"A1", "B2", "C3"}))
 	}
 
-	connFn := func(req byte) (bs net.Conn) {
+	connFn := func() (bs net.Conn) {
 		// log("calling f1")
 		bs, err2 := net.Dial(ln.Addr().Network(), ln.Addr().String())
-		// log("f1. bs: %v, err2: %v", bs, err2)
+		//fmt.Printf("f1. bs: %v, err2: %v\n", bs, err2)
 		checkErrT(t, err2)
-		n1, err2 := bs.Write([]byte{req})
-		checkErrT(t, err2)
-		checkEqualT(t, n1, 1)
 		return
 	}
 
+	exitFn := func() {
+		atomic.StoreUint64(&serverExitFlag, 1)
+		bs := connFn()
+		<-serverExitChan
+		bs.Close()
+		// serverExitChan <- true
+	}
+
 	go serverFn()
+	runtime.Gosched()
+	//time.Sleep(100 * time.Millisecond)
+	if exitSleepMs == 0 {
+		defer ln.Close()
+		defer exitFn()
+	} 
 	if doRequest {
-		bs := connFn('R')
+		bs := connFn()
 		cc := rr.ClientCodec(bs, h)
 		clientFn(cc)
 	}
-	if doExit {
-		bs := connFn('X')
-		<-serverExitChan
-		bs.Close()
-		// serverExitChan <- true
+	if exitSleepMs != 0 {
+		go func() {
+			defer ln.Close()
+			time.Sleep(exitSleepMs)
+			exitFn()
+		}()
 	}
+	return
 }
 
 // Comprehensive testing that generates data encoded from python msgpack,
@@ -769,7 +782,7 @@ func doTestMsgpackPythonGenStreams(t *testing.T) {
 	//cmd.Stdout = &out
 	var cmdout []byte
 	if cmdout, err = cmd.CombinedOutput(); err != nil {
-		logT(t, "-------- Error running python build.py. Err: %v", err)
+		logT(t, "-------- Error running msgpack_test.py testdata. Err: %v", err)
 		logT(t, "         %v", string(cmdout))
 		t.FailNow()
 	}
@@ -835,9 +848,51 @@ func doTestMsgpackPythonGenStreams(t *testing.T) {
 		}
 	}
 	testMsgpackH.MapType = oldMapType
+}
+
+// To test MsgpackSpecRpc, we test 3 scenarios:
+//    - Go Client to Go RPC Service (contained within TestMsgpackRpcSpec)
+//    - Go client to Python RPC Service (contained within doTestMsgpackRpcSpecGoClientToPythonSvc)
+//    - Python Client to Go RPC Service (contained within doTestMsgpackRpcSpecPythonClientToGoSvc)
+// 
+// This allows us test the different calling conventions 
+//    - Go Service requires only one argument
+//    - Python Service allows multiple arguments
+
+func doTestMsgpackRpcSpecGoClientToPythonSvc(t *testing.T) {
+	openPort := "6789"
+	cmd := exec.Command("python", "msgpack_test.py", "rpc-server", openPort, "2")
+	checkErrT(t, cmd.Start())
+	time.Sleep(100 * time.Millisecond) // time for python rpc server to start
+	bs, err2 := net.Dial("tcp", ":" + openPort)
+	checkErrT(t, err2)
+	cc := MsgpackSpecRpc.ClientCodec(bs, testMsgpackH)
+	cl := rpc.NewClientWithCodec(cc)
+	defer cl.Close()
+	var rstr string
+	checkErrT(t, cl.Call("EchoStruct", TestABC{"Aa", "Bb", "Cc"}, &rstr))
+	//checkEqualT(t, rstr, "{'A': 'Aa', 'B': 'Bb', 'C': 'Cc'}")
+	var mArgs MsgpackSpecRpcMultiArgs = []interface{}{"A1", "B2", "C3"}
+	checkErrT(t, cl.Call("Echo123", mArgs, &rstr))
+	checkEqualT(t, rstr, "1:A1 2:B2 3:C3")
+}
 
+func doTestMsgpackRpcSpecPythonClientToGoSvc(t *testing.T) {
+	port := doTestRpcOne(t, MsgpackSpecRpc, testMsgpackH, false, 1*time.Second)
+	//time.Sleep(1000 * time.Millisecond)
+	cmd := exec.Command("python", "msgpack_test.py", "rpc-client-go-service", strconv.Itoa(port))
+	var cmdout []byte
+	var err error
+	if cmdout, err = cmd.CombinedOutput(); err != nil {
+		logT(t, "-------- Error running msgpack_test.py rpc-client-go-service. Err: %v", err)
+		logT(t, "         %v", string(cmdout))
+		t.FailNow()
+	}
+	checkEqualT(t, string(cmdout), 
+		fmt.Sprintf("%#v\n%#v\n", []string{"A1", "B2", "C3"}, TestABC{"Aa", "Bb", "Cc"}))
 }
 
+
 func TestMsgpackCodecsTable(t *testing.T) {
 	testCodecTableOne(t, testMsgpackH)
 }
@@ -855,15 +910,15 @@ func TestBincCodecsMisc(t *testing.T) {
 }
 
 func TestMsgpackRpcGo(t *testing.T) {
-	doTestRpcOne(t, GoRpc, testMsgpackH, true, true, true)
+	doTestRpcOne(t, GoRpc, testMsgpackH, true, 0)
 }
 
 func TestMsgpackRpcSpec(t *testing.T) {
-	doTestRpcOne(t, MsgpackSpecRpc, testMsgpackH, true, true, true)
+	doTestRpcOne(t, MsgpackSpecRpc, testMsgpackH, true, 0)
 }
 
 func TestBincRpcGo(t *testing.T) {
-	doTestRpcOne(t, GoRpc, testBincH, true, true, true)
+	doTestRpcOne(t, GoRpc, testBincH, true, 0)
 }
 
 //TODO: 

codec/decode.go

@@ -21,7 +21,7 @@ type decodeNakedContext uint8
 const (
 	dncHandled decodeNakedContext = iota
 	dncNil
-	dncExt
+	// dncExt
 	dncContainer
 )
 
@@ -42,7 +42,7 @@ const (
 	detTimestamp
 	detExt
 )
-	
+
 // decReader abstracts the reading source, allowing implementations that can
 // read from an io.Reader or directly off a byte slice with zero-copying.
 type decReader interface {
@@ -58,7 +58,8 @@ type decDriver interface {
 	initReadNext()
 	tryDecodeAsNil() bool
 	currentEncodedType() decodeEncodedType
-	decodeBuiltinType(rt uintptr, rv reflect.Value) bool
+	isBuiltinType(rt uintptr) bool
+	decodeBuiltinType(rt uintptr, rv reflect.Value)
 	//decodeNaked should completely handle extensions, builtins, primitives, etc.
 	//Numbers are decoded as int64, uint64, float64 only (no smaller sized number types).
 	decodeNaked() (rv reflect.Value, ctx decodeNakedContext)
@@ -74,11 +75,275 @@ type decDriver interface {
 	readArrayLen() int
 }
 
+// decFnInfo has methods for registering handling decoding of a specific type
+// based on some characteristics (builtin, extension, reflect Kind, etc)
+type decFnInfo struct {
+	sis *typeInfo
+	d   *Decoder
+	dd  decDriver
+	rt    reflect.Type
+	rtid  uintptr
+	xfFn  func(reflect.Value, []byte) error
+	xfTag byte 
+}
+
+type decFn struct {
+	i *decFnInfo
+	f func(*decFnInfo, reflect.Value) 
+}
+
 // A Decoder reads and decodes an object from an input stream in the codec format.
 type Decoder struct {
 	r decReader
 	d decDriver
 	h decodeHandleI
+	f map[uintptr]decFn
+}
+
+func (f *decFnInfo) builtin(rv reflect.Value) {
+	baseRv := rv
+	baseIndir := f.sis.baseIndir
+	for j := int8(0); j < baseIndir; j++ {
+		baseRv = baseRv.Elem()
+	}
+	f.dd.decodeBuiltinType(f.sis.baseId, baseRv)
+}
+
+func (f *decFnInfo) ext(rv reflect.Value) {
+	xbs := f.dd.decodeExt(f.xfTag)
+	baseRv := rv
+	baseIndir := f.sis.baseIndir
+	for j := int8(0); j < baseIndir; j++ {
+		baseRv = baseRv.Elem()
+	}
+	if fnerr := f.xfFn(baseRv, xbs); fnerr != nil {
+		panic(fnerr)
+	}
+}
+
+func (f *decFnInfo) binaryMarshal(rv reflect.Value) {
+	var bm binaryUnmarshaler
+	if f.sis.unmIndir == -1 {
+		bm = rv.Addr().Interface().(binaryUnmarshaler)
+	} else if f.sis.unmIndir == 0 {
+		bm = rv.Interface().(binaryUnmarshaler)
+	} else {
+		rv2 := rv
+		unmIndir := f.sis.unmIndir
+		for j := int8(0); j < unmIndir; j++ {
+			rv2 = rv.Elem()
+		}
+		bm = rv2.Interface().(binaryUnmarshaler)
+	}
+	xbs, _ := f.dd.decodeBytes(nil)
+	if fnerr := bm.UnmarshalBinary(xbs); fnerr != nil {
+		panic(fnerr)
+	}
+}
+
+func (f *decFnInfo) kErr(rv reflect.Value) {
+	decErr("Unhandled value for kind: %v: %s", rv.Kind(), msgBadDesc)
+}
+
+func (f *decFnInfo) kString(rv reflect.Value) {
+	rv.SetString(f.dd.decodeString())
+}
+
+func (f *decFnInfo) kBool(rv reflect.Value) {
+	rv.SetBool(f.dd.decodeBool())
+}
+
+func (f *decFnInfo) kInt(rv reflect.Value) {
+	rv.SetInt(f.dd.decodeInt(intBitsize))
+}
+
+func (f *decFnInfo) kInt64(rv reflect.Value) {
+	rv.SetInt(f.dd.decodeInt(64))
+}
+
+func (f *decFnInfo) kInt32(rv reflect.Value) {
+	rv.SetInt(f.dd.decodeInt(32))
+}
+
+func (f *decFnInfo) kInt8(rv reflect.Value) {
+	rv.SetInt(f.dd.decodeInt(8))
+}
+
+func (f *decFnInfo) kInt16(rv reflect.Value) {
+	rv.SetInt(f.dd.decodeInt(16))
+}
+
+func (f *decFnInfo) kFloat32(rv reflect.Value) {
+	rv.SetFloat(f.dd.decodeFloat(true))
+}
+
+func (f *decFnInfo) kFloat64(rv reflect.Value) {
+	rv.SetFloat(f.dd.decodeFloat(false))
+}
+
+func (f *decFnInfo) kUint8(rv reflect.Value) {
+	rv.SetUint(f.dd.decodeUint(8))
+}
+
+func (f *decFnInfo) kUint64(rv reflect.Value) {
+	rv.SetUint(f.dd.decodeUint(64))
+}
+
+func (f *decFnInfo) kUint(rv reflect.Value) {
+	rv.SetUint(f.dd.decodeUint(uintBitsize))
+}
+
+func (f *decFnInfo) kUint32(rv reflect.Value) {
+	rv.SetUint(f.dd.decodeUint(32))
+}
+
+func (f *decFnInfo) kUint16(rv reflect.Value) {
+	rv.SetUint(f.dd.decodeUint(16))
+}
+
+func (f *decFnInfo) kPtr(rv reflect.Value) {
+	if rv.IsNil() {
+		rv.Set(reflect.New(f.rt.Elem()))
+	}
+	f.d.decodeValue(rv.Elem())
+}
+
+func (f *decFnInfo) kInterface(rv reflect.Value) {
+	f.d.decodeValue(rv.Elem())
+}
+
+func (f *decFnInfo) kStruct(rv reflect.Value) {
+	if currEncodedType := f.dd.currentEncodedType(); currEncodedType == detMap {
+		containerLen := f.dd.readMapLen()
+		if containerLen == 0 {
+			return
+		}
+		sissis := f.sis.sis 
+		for j := 0; j < containerLen; j++ {
+			// var rvkencname string
+			// ddecode(&rvkencname)
+			f.dd.initReadNext()
+			rvkencname := f.dd.decodeString()
+			// rvksi := sis.getForEncName(rvkencname)
+			if k := f.sis.indexForEncName(rvkencname); k > -1 {
+				sfik := sissis[k]
+				if sfik.i != -1 {
+					f.d.decodeValue(rv.Field(int(sfik.i)))
+				} else {
+					f.d.decodeValue(rv.FieldByIndex(sfik.is))
+				}
+				// f.d.decodeValue(sis.field(k, rv))
+			} else {
+				if f.d.h.errorIfNoField() {
+					decErr("No matching struct field found when decoding stream map with key: %v", rvkencname)
+				} else {
+					var nilintf0 interface{}
+					f.d.decodeValue(reflect.ValueOf(&nilintf0).Elem())
+				}
+			}
+		}
+	} else if currEncodedType == detArray {
+		containerLen := f.dd.readArrayLen()
+		if containerLen == 0 {
+			return
+		}
+		for j, si := range f.sis.sisp {
+			if j == containerLen {
+				break
+			}
+			if si.i != -1 {
+				f.d.decodeValue(rv.Field(int(si.i)))
+			} else {
+				f.d.decodeValue(rv.FieldByIndex(si.is))
+			}
+		}
+		if containerLen > len(f.sis.sisp) {
+			// read remaining values and throw away
+			for j := len(f.sis.sisp); j < containerLen; j++ {
+				var nilintf0 interface{}
+				f.d.decodeValue(reflect.ValueOf(&nilintf0).Elem())
+			}
+		}
+	} else {
+		decErr("Only encoded map or array can be decoded into a struct. (decodeEncodedType: %x)", currEncodedType)
+	}
+}
+
+func (f *decFnInfo) kSlice(rv reflect.Value) {
+	// Be more careful calling Set() here, because a reflect.Value from an array
+	// may have come in here (which may not be settable).
+	// In places where the slice got from an array could be, we should guard with CanSet() calls.
+
+	if f.rtid == byteSliceTypId { // rawbytes
+		if bs2, changed2 := f.dd.decodeBytes(rv.Bytes()); changed2 {
+			rv.SetBytes(bs2)
+		}
+		return
+	}
+
+	containerLen := f.dd.readArrayLen()
+
+	if rv.IsNil() {
+		rv.Set(reflect.MakeSlice(f.rt, containerLen, containerLen))
+	} 
+	if containerLen == 0 {
+		return
+	}
+
+	// if we need to reset rv but it cannot be set, we should err out.
+	// for example, if slice is got from unaddressable array, CanSet = false
+	if rvcap, rvlen := rv.Len(), rv.Cap(); containerLen > rvcap {
+		if rv.CanSet() {
+			rvn := reflect.MakeSlice(f.rt, containerLen, containerLen)
+			if rvlen > 0 {
+				reflect.Copy(rvn, rv)
+			}
+			rv.Set(rvn)
+		} else {
+			decErr("Cannot reset slice with less cap: %v than stream contents: %v", rvcap, containerLen)
+		}
+	} else if containerLen > rvlen {
+		rv.SetLen(containerLen)
+	}
+	for j := 0; j < containerLen; j++ {
+		f.d.decodeValue(rv.Index(j))
+	}
+}
+
+func (f *decFnInfo) kArray(rv reflect.Value) {
+	f.d.decodeValue(rv.Slice(0, rv.Len()))
+}
+
+func (f *decFnInfo) kMap(rv reflect.Value) {
+	containerLen := f.dd.readMapLen()
+
+	if rv.IsNil() {
+		rv.Set(reflect.MakeMap(f.rt))
+	}
+	
+	if containerLen == 0 {
+		return
+	}
+
+	ktype, vtype := f.rt.Key(), f.rt.Elem()
+	for j := 0; j < containerLen; j++ {
+		rvk := reflect.New(ktype).Elem()
+		f.d.decodeValue(rvk)
+
+		if ktype == intfTyp {
+			rvk = rvk.Elem()
+			if rvk.Type() == byteSliceTyp {
+				rvk = reflect.ValueOf(string(rvk.Bytes()))
+			}
+		}
+		rvv := rv.MapIndex(rvk)
+		if !rvv.IsValid() {
+			rvv = reflect.New(vtype).Elem()
+		}
+
+		f.d.decodeValue(rvv)
+		rv.SetMapIndex(rvk, rvv)
+	}
 }
 
 // ioDecReader is a decReader that reads off an io.Reader
@@ -95,28 +360,11 @@ type bytesDecReader struct {
 	a int    // available
 }
 
-type decExtTagFn struct {
-	fn  func(reflect.Value, []byte) error
-	tag byte
-}
-
-type decExtTypeTagFn struct {
-	rtid uintptr
-	rt reflect.Type
-	decExtTagFn
-}
-
 type decodeHandleI interface {
 	getDecodeExt(rt uintptr) (tag byte, fn func(reflect.Value, []byte) error)
 	errorIfNoField() bool
 }
 
-type decHandle struct {
-	// put word-aligned fields first (before bools, etc)
-	exts     []decExtTypeTagFn
-	extFuncs map[uintptr]decExtTagFn
-}
-
 type DecodeOptions struct {
 	// An instance of MapType is used during schema-less decoding of a map in the stream.
 	// If nil, we use map[interface{}]interface{}
@@ -133,62 +381,6 @@ func (o *DecodeOptions) errorIfNoField() bool {
 	return o.ErrorIfNoField
 }
 
-// addDecodeExt registers a function to handle decoding into a given type when an
-// extension type and specific tag byte is detected in the codec stream.
-// To remove an extension, pass fn=nil.
-func (o *decHandle) addDecodeExt(rtid uintptr, rt reflect.Type, tag byte, fn func(reflect.Value, []byte) error) {
-	if o.exts == nil {
-		o.exts = make([]decExtTypeTagFn, 0, 2)
-		o.extFuncs = make(map[uintptr]decExtTagFn, 2)
-	} else {
-		if _, ok := o.extFuncs[rtid]; ok {
-			delete(o.extFuncs, rtid)
-			for i := 0; i < len(o.exts); i++ {
-				if o.exts[i].rtid == rtid {
-					o.exts = append(o.exts[:i], o.exts[i+1:]...)
-					break
-				}
-			}
-		}
-	}
-	if fn != nil {
-		o.extFuncs[rtid] = decExtTagFn{fn, tag}
-		o.exts = append(o.exts, decExtTypeTagFn{rtid, rt, decExtTagFn{fn, tag}})
-	}
-}
-
-func (o *decHandle) getDecodeExtForTag(tag byte) (rv reflect.Value, fn func(reflect.Value, []byte) error) {
-	for i, l := 0, len(o.exts); i < l; i++ {
-		if o.exts[i].tag == tag {
-			if o.exts[i].rt.Kind() == reflect.Ptr {
-				rv = reflect.New(o.exts[i].rt.Elem())
-			} else {
-				rv = reflect.New(o.exts[i].rt).Elem()
-			}
-			fn = o.exts[i].fn
-			return 
-		}
-	}
-	return
-}
-
-func (o *decHandle) getDecodeExt(rt uintptr) (tag byte, fn func(reflect.Value, []byte) error) {
-	if l := len(o.exts); l == 0 {
-		return
-	} else if l < mapAccessThreshold {
-		for i := 0; i < l; i++ {
-			if o.exts[i].rtid == rt {
-				x := o.exts[i].decExtTagFn
-				return x.tag, x.fn
-			}
-		}
-	} else {
-		x := o.extFuncs[rt]
-		return x.tag, x.fn
-	}
-	return
-}
-
 // NewDecoder returns a Decoder for decoding a stream of bytes from an io.Reader.
 // 
 // For efficiency, Users are encouraged to pass in a memory buffered writer
@@ -234,6 +426,12 @@ func NewDecoderBytes(in []byte, h Handle) *Decoder {
 // are decoded appropriately (e.g. bool), and configurations exist on the Handle to override
 // defaults (e.g. for MapType, SliceType and how to decode raw bytes).
 // 
+// When decoding into a non-nil interface{} value, the mode of encoding is based on the 
+// type of the value. When a value is seen:
+//   - If an extension is registered for it, call that extension function
+//   - If it implements BinaryUnmarshaler, call its UnmarshalBinary(data []byte) error
+//   - Else decode it based on its reflect.Kind
+// 
 // There are some special rules when decoding into containers (slice/array/map/struct).
 // Decode will typically use the stream contents to UPDATE the container. 
 //   - This means that for a struct or map, we just update matching fields or keys.
@@ -298,42 +496,34 @@ func (d *Decoder) decode(iv interface{}) {
 }
 
 func (d *Decoder) decodeValue(rv reflect.Value) {
-	// Note: if stream is set to nil, we set the corresponding value to its "zero" value
-
-	// var ctr int (define this above the  function if trying to do this run)
-	// ctr++
-	// log(".. [%v] enter decode: rv: %v <==> %T <==> %v", ctr, rv, rv.Interface(), rv.Interface())
-	// defer func(ctr2 int) {
-	// 	log(".... [%v] exit decode: rv: %v <==> %T <==> %v", ctr2, rv, rv.Interface(), rv.Interface())
-	// }(ctr)
-	dd := d.d //so we don't dereference constantly
-	dd.initReadNext()
+	d.d.initReadNext()
 
-	rvOrig := rv
-	wasNilIntf := rv.Kind() == reflect.Interface && rv.IsNil()
 	rt := rv.Type()
+	rvOrig := rv
+	wasNilIntf := rt.Kind() == reflect.Interface && rv.IsNil()
 
 	var ndesc decodeNakedContext
 	//if nil interface, use some hieristics to set the nil interface to an
 	//appropriate value based on the first byte read (byte descriptor bd)
 	if wasNilIntf {
-		if dd.tryDecodeAsNil() {
+		// e.g. nil interface{}, error, io.Reader, etc
+		rv, ndesc = d.d.decodeNaked()
+		if ndesc == dncNil {
 			return
 		}
-		//Prevent from decoding into e.g. error, io.Reader, etc if it's nil and non-nil value in stream.
+		//Prevent from decoding into nil error, io.Reader, etc if non-nil value in stream.
 		//We can only decode into interface{} (0 methods). Else reflect.Set fails later.
 		if num := rt.NumMethod(); num > 0 {
 			decErr("decodeValue: Cannot decode non-nil codec value into nil %v (%v methods)", rt, num)
-		} else {
-			rv, ndesc = dd.decodeNaked()
-			if ndesc == dncHandled {
-				rvOrig.Set(rv)
-				return
-			}
-			rt = rv.Type()
+		} 
+		if ndesc == dncHandled {
+			rvOrig.Set(rv)
+			return
 		}
-	} else if dd.tryDecodeAsNil() {
-		// Note: if stream is set to nil, we set the dereferenced value to its "zero" value (if settable).
+		rt = rv.Type()
+	} else if d.d.tryDecodeAsNil() {
+		// Note: if value in stream is nil, 
+		// then we set the dereferenced value to its "zero" value (if settable).
 		for rv.Kind() == reflect.Ptr {
 			rv = rv.Elem()
 		}
@@ -345,222 +535,86 @@ func (d *Decoder) decodeValue(rv reflect.Value) {
 
 	rtid := reflect.ValueOf(rt).Pointer()
 	
-	// An extension can be registered for any type, regardless of the Kind
-	// (e.g. type BitSet int64, type MyStruct { / * unexported fields * / }, type X []int, etc.
-	//
-	// We can't check if it's an extension byte here first, because the user may have
-	// registered a pointer or non-pointer type, meaning we may have to recurse first
-	// before matching a mapped type, even though the extension byte is already detected.
-	//
-	// If we are checking for builtin or ext type here, it means we didn't go through decodeNaked,
-	// Because decodeNaked would have handled it. It also means wasNilIntf = false.
-	if dd.decodeBuiltinType(rtid, rv) {
-		return
-	}
+	// retrieve or register a focus'ed function for this type
+	// to eliminate need to do the retrieval multiple times
 	
-	if bfnTag, bfnFn := d.h.getDecodeExt(rtid); bfnFn != nil {
-		xbs := dd.decodeExt(bfnTag)
-		if fnerr := bfnFn(rv, xbs); fnerr != nil {
-			panic(fnerr)
-		}
-		return
+	if d.f == nil {
+		// debugf("---->Creating new dec f map for type: %v\n", rt)
+		d.f = make(map[uintptr]decFn, 16)
 	}
-
-	// TODO: Decode if type is an encoding.BinaryUnmarshaler: UnmarshalBinary(data []byte) error
-	// There is a cost, as we need to change the rv to an interface{} first.
-	
-	// NOTE: if decoding into a nil interface{}, we return a non-nil
-	// value except even if the container registers a length of 0.
-	//
-	// NOTE: Do not make blocks for struct, slice, map, etc individual methods.
-	// It ends up being more expensive, because they recursively calls decodeValue
-	//
-	// (Mar 7, 2013. DON'T REARRANGE ... code clarity)
-	// tried arranging in sequence of most probable ones.
-	// string, bool, integer, float, struct, ptr, slice, array, map, interface, uint.
-	switch rk := rv.Kind(); rk {
-	case reflect.String:
-		rv.SetString(dd.decodeString())
-	case reflect.Bool:
-		rv.SetBool(dd.decodeBool())
-	case reflect.Int:
-		rv.SetInt(dd.decodeInt(intBitsize))
-	case reflect.Int64:
-		rv.SetInt(dd.decodeInt(64))
-	case reflect.Int32:
-		rv.SetInt(dd.decodeInt(32))
-	case reflect.Int8:
-		rv.SetInt(dd.decodeInt(8))
-	case reflect.Int16:
-		rv.SetInt(dd.decodeInt(16))
-	case reflect.Float32:
-		rv.SetFloat(dd.decodeFloat(true))
-	case reflect.Float64:
-		rv.SetFloat(dd.decodeFloat(false))
-	case reflect.Uint8:
-		rv.SetUint(dd.decodeUint(8))
-	case reflect.Uint64:
-		rv.SetUint(dd.decodeUint(64))
-	case reflect.Uint:
-		rv.SetUint(dd.decodeUint(uintBitsize))
-	case reflect.Uint32:
-		rv.SetUint(dd.decodeUint(32))
-	case reflect.Uint16:
-		rv.SetUint(dd.decodeUint(16))
-	case reflect.Ptr:
-		if rv.IsNil() {
-			if wasNilIntf {
-				rv = reflect.New(rt.Elem())
-			} else {
-				rv.Set(reflect.New(rt.Elem()))
-			}
-		}
-		d.decodeValue(rv.Elem())
-	case reflect.Interface:
-		d.decodeValue(rv.Elem())
-	case reflect.Struct:
-		if currEncodedType := dd.currentEncodedType(); currEncodedType == detMap {
-			containerLen := dd.readMapLen()
-			if containerLen == 0 {
-				break
-			}
-			sfi := getStructFieldInfos(rtid, rt)
-			sissis := sfi.sis 
-			for j := 0; j < containerLen; j++ {
-				// var rvkencname string
-				// ddecode(&rvkencname)
-				dd.initReadNext()
-				rvkencname := dd.decodeString()
-				// rvksi := sfi.getForEncName(rvkencname)
-				if k := sfi.indexForEncName(rvkencname); k > -1 {
-					sfik := sissis[k]
-					if sfik.i != -1 {
-						d.decodeValue(rv.Field(int(sfik.i)))
-					} else {
-						d.decodeValue(rv.FieldByIndex(sfik.is))
-					}
-					// d.decodeValue(sfi.field(k, rv))
-				} else {
-					if d.h.errorIfNoField() {
-						decErr("No matching struct field found when decoding stream map with key: %v", rvkencname)
-					} else {
-						var nilintf0 interface{}
-						d.decodeValue(reflect.ValueOf(&nilintf0).Elem())
-					}
-				}
-			}
-		} else if currEncodedType == detArray {
-			containerLen := dd.readArrayLen()
-			if containerLen == 0 {
-				break
-			}
-			sfi := getStructFieldInfos(rtid, rt)
-			for j, si := range sfi.sisp {
-				if j == containerLen {
-					break
-				}
-				if si.i != -1 {
-					d.decodeValue(rv.Field(int(si.i)))
-				} else {
-					d.decodeValue(rv.FieldByIndex(si.is))
-				}
-			}
-			if containerLen > len(sfi.sisp) {
-				// read remaining values and throw away
-				for j := len(sfi.sisp); j < containerLen; j++ {
-					var nilintf0 interface{}
-					d.decodeValue(reflect.ValueOf(&nilintf0).Elem())
-				}
-			}
+	fn, ok := d.f[rtid]
+	if !ok {
+		// debugf("\tCreating new dec fn for type: %v\n", rt)
+		fi := decFnInfo { sis:getTypeInfo(rtid, rt), d:d, dd:d.d, rt:rt, rtid:rtid }
+		// An extension can be registered for any type, regardless of the Kind
+		// (e.g. type BitSet int64, type MyStruct { / * unexported fields * / }, type X []int, etc.
+		//
+		// We can't check if it's an extension byte here first, because the user may have
+		// registered a pointer or non-pointer type, meaning we may have to recurse first
+		// before matching a mapped type, even though the extension byte is already detected.
+		//
+		// If we are checking for builtin or ext type here, it means we didn't go through decodeNaked,
+		// Because decodeNaked would have handled it. It also means wasNilIntf = false.
+		if d.d.isBuiltinType(fi.sis.baseId) {
+			fn = decFn { &fi, (*decFnInfo).builtin }
+		} else if xfTag, xfFn := d.h.getDecodeExt(fi.sis.baseId); xfFn != nil {
+			fi.xfTag, fi.xfFn = xfTag, xfFn
+			fn = decFn { &fi, (*decFnInfo).ext }
+		} else if supportBinaryMarshal && fi.sis.unm {
+			fn = decFn { &fi, (*decFnInfo).binaryMarshal }
 		} else {
-			decErr("Only encoded map or array can be decoded into a struct")
-		}
-		
-	case reflect.Slice:
-		// Be more careful calling Set() here, because a reflect.Value from an array
-		// may have come in here (which may not be settable).
-		// In places where the slice got from an array could be, we should guard with CanSet() calls.
-
-		if rt == byteSliceTyp { // rawbytes
-			if bs2, changed2 := dd.decodeBytes(rv.Bytes()); changed2 {
-				rv.SetBytes(bs2)
-			}
-			if wasNilIntf && rv.IsNil() {
-				rv.SetBytes([]byte{})
+			// NOTE: if decoding into a nil interface{}, we return a non-nil
+			// value except even if the container registers a length of 0.
+			switch rk := rt.Kind(); rk {
+			case reflect.String:
+				fn = decFn { &fi, (*decFnInfo).kString }
+			case reflect.Bool:
+				fn = decFn { &fi, (*decFnInfo).kBool }
+			case reflect.Int:
+				fn = decFn { &fi, (*decFnInfo).kInt }
+			case reflect.Int64:
+				fn = decFn { &fi, (*decFnInfo).kInt64 }
+			case reflect.Int32:
+				fn = decFn { &fi, (*decFnInfo).kInt32 }
+			case reflect.Int8:
+				fn = decFn { &fi, (*decFnInfo).kInt8 }
+			case reflect.Int16:
+				fn = decFn { &fi, (*decFnInfo).kInt16 }
+			case reflect.Float32:
+				fn = decFn { &fi, (*decFnInfo).kFloat32 }
+			case reflect.Float64:
+				fn = decFn { &fi, (*decFnInfo).kFloat64 }
+			case reflect.Uint8:
+				fn = decFn { &fi, (*decFnInfo).kUint8 }
+			case reflect.Uint64:
+				fn = decFn { &fi, (*decFnInfo).kUint64 }
+			case reflect.Uint:
+				fn = decFn { &fi, (*decFnInfo).kUint }
+			case reflect.Uint32:
+				fn = decFn { &fi, (*decFnInfo).kUint32 }
+			case reflect.Uint16:
+				fn = decFn { &fi, (*decFnInfo).kUint16 }
+			case reflect.Ptr:
+				fn = decFn { &fi, (*decFnInfo).kPtr }
+			case reflect.Interface:
+				fn = decFn { &fi, (*decFnInfo).kInterface }
+			case reflect.Struct:
+				fn = decFn { &fi, (*decFnInfo).kStruct }
+			case reflect.Slice:
+				fn = decFn { &fi, (*decFnInfo).kSlice }
+			case reflect.Array:
+				fn = decFn { &fi, (*decFnInfo).kArray }
+			case reflect.Map:
+				fn = decFn { &fi, (*decFnInfo).kMap }
+			default:
+				fn = decFn { &fi, (*decFnInfo).kErr }
 			}
-			break
-		}
-
-		containerLen := dd.readArrayLen()
-
-		if wasNilIntf {
-			rv = reflect.MakeSlice(rt, containerLen, containerLen)
-		}
-		if containerLen == 0 {
-			if rv.IsNil() {
-				rv.Set(reflect.MakeSlice(rt, containerLen, containerLen))
-			} 
-			break
-		}
-
-		if rv.IsNil() {
-			// wasNilIntf only applies if rv is nil (since that's what we did earlier)
-			rv.Set(reflect.MakeSlice(rt, containerLen, containerLen))
-		} else {
-			// if we need to reset rv but it cannot be set, we should err out.
-			// for example, if slice is got from unaddressable array, CanSet = false
-			if rvcap, rvlen := rv.Len(), rv.Cap(); containerLen > rvcap {
-				if rv.CanSet() {
-					rvn := reflect.MakeSlice(rt, containerLen, containerLen)
-					if rvlen > 0 {
-						reflect.Copy(rvn, rv)
-					}
-					rv.Set(rvn)
-				} else {
-					decErr("Cannot reset slice with less cap: %v that stream contents: %v", rvcap, containerLen)
-				}
-			} else if containerLen > rvlen {
-				rv.SetLen(containerLen)
-			}
-		}
-		for j := 0; j < containerLen; j++ {
-			d.decodeValue(rv.Index(j))
-		}
-	case reflect.Array:
-		d.decodeValue(rv.Slice(0, rv.Len()))
-	case reflect.Map:
-		containerLen := dd.readMapLen()
-
-		if rv.IsNil() {
-			rv.Set(reflect.MakeMap(rt))
-		}
-		
-		if containerLen == 0 {
-			break
-		}
-
-		ktype, vtype := rt.Key(), rt.Elem()
-		for j := 0; j < containerLen; j++ {
-			rvk := reflect.New(ktype).Elem()
-			d.decodeValue(rvk)
-
-			if ktype == intfTyp {
-				rvk = rvk.Elem()
-				if rvk.Type() == byteSliceTyp {
-					rvk = reflect.ValueOf(string(rvk.Bytes()))
-				}
-			}
-			rvv := rv.MapIndex(rvk)
-			if !rvv.IsValid() {
-				rvv = reflect.New(vtype).Elem()
-			}
-
-			d.decodeValue(rvv)
-			rv.SetMapIndex(rvk, rvv)
-		}
-	default:
-		decErr("Unhandled value for kind: %v: %s", rk, msgBadDesc)
+		}		
+		d.f[rtid] = fn
 	}
+	
+	fn.f(fn.i, rv)
+	
 
 	if wasNilIntf {
 		rvOrig.Set(rv)

codec/encode.go

@@ -30,8 +30,10 @@ type encWriter interface {
 	atEndOfEncode()
 }
 
+// encDriver abstracts the actual codec (binc vs msgpack, etc)
 type encDriver interface {
-	encodeBuiltinType(rt uintptr, rv reflect.Value) bool
+	isBuiltinType(rt uintptr) bool
+	encodeBuiltinType(rt uintptr, rv reflect.Value)
 	encodeNil()
 	encodeInt(i int64)
 	encodeUint(i uint64)
@@ -49,17 +51,38 @@ type encDriver interface {
 	//encStringRunes(c charEncoding, v []rune)
 }
 
+// encodeHandleI is the interface that the encode functions need.
 type encodeHandleI interface {
 	getEncodeExt(rt uintptr) (tag byte, fn func(reflect.Value) ([]byte, error))
 	writeExt() bool
 	structToArray() bool
 }
 
+type encFnInfo struct {
+	sis   *typeInfo
+	e     *Encoder
+	ee    encDriver
+	rt    reflect.Type
+	rtid  uintptr
+	xfFn  func(reflect.Value) ([]byte, error)
+	xfTag byte 
+}
+
+// encFn encapsulates the captured variables and the encode function.
+// This way, we only do some calculations one times, and pass to the 
+// code block that should be called (encapsulated in a function) 
+// instead of executing the checks every time.
+type encFn struct {
+	i *encFnInfo
+	f func(*encFnInfo, reflect.Value)
+}
+
 // An Encoder writes an object to an output stream in the codec format.
 type Encoder struct {
 	w encWriter
 	e encDriver
 	h encodeHandleI
+	f map[uintptr]encFn
 }
 
 type ioEncWriterWriter interface {
@@ -92,22 +115,6 @@ type bytesEncWriter struct {
 	out *[]byte // write out on atEndOfEncode
 }
 
-type encExtTagFn struct {
-	fn  func(reflect.Value) ([]byte, error)
-	tag byte
-}
-
-type encExtTypeTagFn struct {
-	rt uintptr
-	encExtTagFn
-}
-
-// EncoderOptions contain options for the encoder, e.g. registered extension functions.
-type encHandle struct {
-	extFuncs map[uintptr]encExtTagFn
-	exts     []encExtTypeTagFn
-}
-
 type EncodeOptions struct {
 	// Encode a struct as an array, and not as a map.
 	StructToArray bool
@@ -133,51 +140,210 @@ func (o *simpleIoEncWriterWriter) Write(p []byte) (n int, err error) {
 }
 
 
-// addEncodeExt registers a function to handle encoding a given type as an extension
-// with a specific specific tag byte.
-// To remove an extension, pass fn=nil.
-func (o *encHandle) addEncodeExt(rt uintptr, tag byte, fn func(reflect.Value) ([]byte, error)) {
-	if o.exts == nil {
-		o.exts = make([]encExtTypeTagFn, 0, 8)
-		o.extFuncs = make(map[uintptr]encExtTagFn, 8)
+func (o *EncodeOptions) structToArray() bool {
+	return o.StructToArray
+}
+
+func (f *encFnInfo) builtin(rv reflect.Value) {
+	baseRv := rv
+	for j := int8(0); j < f.sis.baseIndir; j++ {
+		baseRv = baseRv.Elem()
+	}
+	f.ee.encodeBuiltinType(f.sis.baseId, baseRv)
+}
+
+func (f *encFnInfo) ext(rv reflect.Value) {
+	baseRv := rv
+	for j := int8(0); j < f.sis.baseIndir; j++ {
+		baseRv = baseRv.Elem()
+	}
+	bs, fnerr := f.xfFn(baseRv)
+	if fnerr != nil {
+		panic(fnerr)
+	}
+	if bs == nil {
+		f.ee.encodeNil()
+		return
+	}
+	if f.e.h.writeExt() {
+		f.ee.encodeExtPreamble(f.xfTag, len(bs))
+		f.e.w.writeb(bs)
 	} else {
-		if _, ok := o.extFuncs[rt]; ok {
-			delete(o.extFuncs, rt)
-			for i := 0; i < len(o.exts); i++ {
-				if o.exts[i].rt == rt {
-					o.exts = append(o.exts[:i], o.exts[i+1:]...)
-					break
-				}
-			}
+		f.ee.encodeStringBytes(c_RAW, bs)
+	}
+	
+}
+
+func (f *encFnInfo) binaryMarshal(rv reflect.Value) {
+	var bm binaryMarshaler
+	if f.sis.mIndir == 0 {
+		bm = rv.Interface().(binaryMarshaler)
+	} else if f.sis.mIndir == -1 {
+		bm = rv.Addr().Interface().(binaryMarshaler)
+	} else {
+		rv2 := rv
+		for j := int8(0); j < f.sis.mIndir; j++ {
+			rv2 = rv.Elem()
 		}
+		bm = rv2.Interface().(binaryMarshaler)
 	}
-	if fn != nil {
-		o.extFuncs[rt] = encExtTagFn{fn, tag}
-		o.exts = append(o.exts, encExtTypeTagFn{rt, encExtTagFn{fn, tag}})
+	// debugf(">>>> binaryMarshaler: %T", rv.Interface())
+	bs, fnerr := bm.MarshalBinary()
+	if fnerr != nil {
+		panic(fnerr)
 	}
+	if bs == nil {
+		f.ee.encodeNil()
+	} else {
+		f.ee.encodeStringBytes(c_RAW, bs)
+	}
+
 }
 
-func (o *encHandle) getEncodeExt(rt uintptr) (tag byte, fn func(reflect.Value) ([]byte, error)) {
-	if l := len(o.exts); l == 0 {
+func (f *encFnInfo) kBool(rv reflect.Value) {
+	f.ee.encodeBool(rv.Bool())
+}
+
+func (f *encFnInfo) kString(rv reflect.Value) {
+	f.ee.encodeString(c_UTF8, rv.String())
+}
+
+func (f *encFnInfo) kFloat64(rv reflect.Value) {
+	f.ee.encodeFloat64(rv.Float())
+}
+
+func (f *encFnInfo) kFloat32(rv reflect.Value) {
+	f.ee.encodeFloat32(float32(rv.Float()))
+}
+
+func (f *encFnInfo) kInt(rv reflect.Value) {
+	f.ee.encodeInt(rv.Int())
+}
+
+func (f *encFnInfo) kUint(rv reflect.Value) {
+	f.ee.encodeUint(rv.Uint())
+}
+
+func (f *encFnInfo) kInvalid(rv reflect.Value) {
+	f.ee.encodeNil()
+}
+
+func (f *encFnInfo) kErr(rv reflect.Value) {
+	encErr("Unsupported kind: %s, for: %#v", rv.Kind(), rv)
+}
+
+func (f *encFnInfo) kSlice(rv reflect.Value) {
+	if rv.IsNil() {
+		f.ee.encodeNil()
 		return
-	} else if l < mapAccessThreshold {
-		for i := 0; i < l; i++ {
-			if o.exts[i].rt == rt {
-				x := o.exts[i].encExtTagFn
-				return x.tag, x.fn
+	}
+	if f.rtid == byteSliceTypId {
+		f.ee.encodeStringBytes(c_RAW, rv.Bytes())
+		return
+	}
+	l := rv.Len()
+	f.ee.encodeArrayPreamble(l)
+	if l == 0 {
+		return
+	}
+	for j := 0; j < l; j++ {
+		f.e.encodeValue(rv.Index(j))
+	}
+}
+
+func (f *encFnInfo) kArray(rv reflect.Value) {
+	f.e.encodeValue(rv.Slice(0, rv.Len()))
+}
+
+func (f *encFnInfo) kStruct(rv reflect.Value) {
+	newlen := len(f.sis.sis)
+	rvals := make([]reflect.Value, newlen)
+	var encnames []string
+	e := f.e
+	sissis := f.sis.sisp
+	toMap := !(f.sis.toArray || e.h.structToArray())
+	// if toMap, use the sorted array. If toArray, use unsorted array (to match sequence in struct)
+	if toMap {
+		sissis = f.sis.sis
+		encnames = make([]string, newlen)
+	}
+	newlen = 0
+	for _, si := range sissis {
+		if si.i != -1 {
+			rvals[newlen] = rv.Field(int(si.i))
+		} else {
+			rvals[newlen] = rv.FieldByIndex(si.is)
+		}
+		if toMap {
+			if si.omitEmpty && isEmptyValue(rvals[newlen]) {
+				continue
+			}
+			encnames[newlen] = si.encName
+		} else {
+			if si.omitEmpty && isEmptyValue(rvals[newlen]) {
+				rvals[newlen] = reflect.Value{} //encode as nil
 			}
 		}
+		newlen++
+	}
+
+	if toMap {
+		ee := f.ee //don't dereference everytime
+		ee.encodeMapPreamble(newlen)
+		for j := 0; j < newlen; j++ {
+			ee.encodeSymbol(encnames[j])
+			e.encodeValue(rvals[j])
+		}
 	} else {
-		x := o.extFuncs[rt]
-		return x.tag, x.fn
+		f.ee.encodeArrayPreamble(newlen)
+		for j := 0; j < newlen; j++ {
+			e.encodeValue(rvals[j])
+		}
 	}
-	return
 }
 
-func (o *EncodeOptions) structToArray() bool {
-	return o.StructToArray
+func (f *encFnInfo) kPtr(rv reflect.Value) {
+	if rv.IsNil() {
+		f.ee.encodeNil()
+		return
+	}
+	f.e.encodeValue(rv.Elem())
+}
+
+func (f *encFnInfo) kInterface(rv reflect.Value) {
+	if rv.IsNil() {
+		f.ee.encodeNil()
+		return
+	}
+	f.e.encodeValue(rv.Elem())
+}
+
+func (f *encFnInfo) kMap(rv reflect.Value) {
+	if rv.IsNil() {
+		f.ee.encodeNil()
+		return
+	}
+	l := rv.Len()
+	f.ee.encodeMapPreamble(l)
+	if l == 0 {
+		return
+	}
+	keyTypeIsString := f.rt.Key().Kind() == reflect.String
+	mks := rv.MapKeys()
+	// for j, lmks := 0, len(mks); j < lmks; j++ {
+	for j := range mks {
+		if keyTypeIsString {
+			f.ee.encodeSymbol(mks[j].String())
+		} else {
+			f.e.encodeValue(mks[j])
+		}
+		f.e.encodeValue(rv.MapIndex(mks[j]))
+	}
+
 }
 
+
+
 // NewEncoder returns an Encoder for encoding into an io.Writer.
 // 
 // For efficiency, Users are encouraged to pass in a memory buffered writer
@@ -194,7 +360,7 @@ func NewEncoder(w io.Writer, h Handle) *Encoder {
 	z := ioEncWriter{
 		w: ww,
 	}
-	return &Encoder{w: &z, h: h, e: h.newEncDriver(&z)}
+	return &Encoder{w: &z, h: h, e: h.newEncDriver(&z) }
 }
 
 // NewEncoderBytes returns an encoder for encoding directly and efficiently
@@ -211,7 +377,7 @@ func NewEncoderBytes(out *[]byte, h Handle) *Encoder {
 		b:   in,
 		out: out,
 	}
-	return &Encoder{w: &z, h: h, e: h.newEncDriver(&z)}
+	return &Encoder{w: &z, h: h, e: h.newEncDriver(&z) }
 }
 
 // Encode writes an object into a stream in the codec format.
@@ -257,10 +423,14 @@ func NewEncoderBytes(out *[]byte, h Handle) *Encoder {
 //                                                         //and encode struct as an array
 //      }   
 //
-// Note:
-//   - Encode will treat struct field names and keys in map[string]XXX as symbols.
-//     Some formats support symbols (e.g. binc) and will properly encode the string
-//     only once in the stream, and use a tag to refer to it thereafter.
+// The mode of encoding is based on the type of the value. When a value is seen:
+//   - If an extension is registered for it, call that extension function
+//   - If it implements BinaryMarshaler, call its MarshalBinary() (data []byte, err error)
+//   - Else encode it based on its reflect.Kind
+// 
+// Note that struct field names and keys in map[string]XXX will be treated as symbols.
+// Some formats support symbols (e.g. binc) and will properly encode the string
+// only once in the stream, and use a tag to refer to it thereafter. 
 func (e *Encoder) Encode(v interface{}) (err error) {
 	defer panicToErr(&err)
 	e.encode(v)
@@ -344,159 +514,61 @@ func (e *Encoder) encodeValue(rv reflect.Value) {
 	rt := rv.Type()
 	rtid := reflect.ValueOf(rt).Pointer()
 	
-	//encode based on type first, since over-rides are based on type.
-	ee := e.e //don't dereference everytime
-	if ee.encodeBuiltinType(rtid, rv) {
-		return
-	}
-
-	//Note: tagFn must handle returning nil if value should be encoded as a nil.
-	if xfTag, xfFn := e.h.getEncodeExt(rtid); xfFn != nil {
-		bs, fnerr := xfFn(rv)
-		if fnerr != nil {
-			panic(fnerr)
-		}
-		if bs == nil {
-			ee.encodeNil()
-			return
-		}
-		if e.h.writeExt() {
-			ee.encodeExtPreamble(xfTag, len(bs))
-			e.w.writeb(bs)
-		} else {
-			ee.encodeStringBytes(c_RAW, bs)
-		}
-		return
-	}
-	
-	// TODO: Encode if type is an encoding.BinaryMarshaler: MarshalBinary() (data []byte, err error)
-	// There is a cost, as we need to change the rv to an interface{} first.
-
-	// ensure more common cases appear early in switch.
-	rk := rv.Kind()
-	switch rk {
-	case reflect.Bool:
-		ee.encodeBool(rv.Bool())
-	case reflect.String:
-		ee.encodeString(c_UTF8, rv.String())
-	case reflect.Float64:
-		ee.encodeFloat64(rv.Float())
-	case reflect.Float32:
-		ee.encodeFloat32(float32(rv.Float()))
-	case reflect.Slice:
-		if rv.IsNil() {
-			ee.encodeNil()
-			break
-		}
-		if rt == byteSliceTyp {
-			ee.encodeStringBytes(c_RAW, rv.Bytes())
-			break
-		}
-		l := rv.Len()
-		ee.encodeArrayPreamble(l)
-		if l == 0 {
-			break
-		}
-		for j := 0; j < l; j++ {
-			e.encodeValue(rv.Index(j))
-		}
-	case reflect.Array:
-		e.encodeValue(rv.Slice(0, rv.Len()))
-	case reflect.Map:
-		if rv.IsNil() {
-			ee.encodeNil()
-			break
-		}
-		l := rv.Len()
-		ee.encodeMapPreamble(l)
-		if l == 0 {
-			break
-		}
-		keyTypeIsString := rt.Key().Kind() == reflect.String
-		mks := rv.MapKeys()
-		// for j, lmks := 0, len(mks); j < lmks; j++ {
-		for j := range mks {
-			if keyTypeIsString {
-				ee.encodeSymbol(mks[j].String())
-			} else {
-				e.encodeValue(mks[j])
-			}
-			e.encodeValue(rv.MapIndex(mks[j]))
-		}
-	case reflect.Struct:
-		sis := getStructFieldInfos(rtid, rt)
-		e.encStruct(sis, rv)
-	case reflect.Ptr:
-		if rv.IsNil() {
-			ee.encodeNil()
-			break
-		}
-		e.encodeValue(rv.Elem())
-	case reflect.Interface:
-		if rv.IsNil() {
-			ee.encodeNil()
-			break
-		}
-		e.encodeValue(rv.Elem())
-	case reflect.Int, reflect.Int8, reflect.Int64, reflect.Int32, reflect.Int16:
-		ee.encodeInt(rv.Int())
-	case reflect.Uint8, reflect.Uint64, reflect.Uint, reflect.Uint32, reflect.Uint16:
-		ee.encodeUint(rv.Uint())
-	case reflect.Invalid:
-		ee.encodeNil()
-	default:
-		encErr("Unsupported kind: %s, for: %#v", rk, rv)
+	if e.f == nil {
+		// debugf("---->Creating new enc f map for type: %v\n", rt)
+		e.f = make(map[uintptr]encFn, 16)
 	}
-	return
-}
 
-func (e *Encoder) encStruct(sis *structFieldInfos, rv reflect.Value) {
-	newlen := len(sis.sis)
-	rvals := make([]reflect.Value, newlen)
-	var encnames []string
-	toMap := !(sis.toArray || e.h.structToArray())
-	sissis := sis.sisp
-	// if toMap, use the sorted array. If toArray, use unsorted array (to match sequence in struct)
-	if toMap {
-		sissis = sis.sis
-		encnames = make([]string, newlen)
-	}
-	newlen = 0
-	for _, si := range sissis {
-		if si.i != -1 {
-			rvals[newlen] = rv.Field(int(si.i))
-		} else {
-			rvals[newlen] = rv.FieldByIndex(si.is)
-		}
-		if toMap {
-			if si.omitEmpty && isEmptyValue(rvals[newlen]) {
-				continue
-			}
-			encnames[newlen] = si.encName
+	fn, ok := e.f[rtid]
+	if !ok {
+		// debugf("\tCreating new enc fn for type: %v\n", rt)
+		fi := encFnInfo { sis:getTypeInfo(rtid, rt), e:e, ee:e.e, rt:rt, rtid:rtid }
+		if e.e.isBuiltinType(fi.sis.baseId) {
+			fn = encFn{ &fi, (*encFnInfo).builtin }
+		} else if xfTag, xfFn := e.h.getEncodeExt(fi.sis.baseId); xfFn != nil {
+			fi.xfTag, fi.xfFn = xfTag, xfFn
+			fn = encFn{ &fi, (*encFnInfo).ext }
+		} else if supportBinaryMarshal && fi.sis.m {
+			fn = encFn{ &fi, (*encFnInfo).binaryMarshal }
 		} else {
-			if si.omitEmpty && isEmptyValue(rvals[newlen]) {
-				rvals[newlen] = reflect.Value{} //encode as nil
+			switch rk := rt.Kind(); rk {
+			case reflect.Bool:
+				fn = encFn{ &fi, (*encFnInfo).kBool }
+			case reflect.String:
+				fn = encFn{ &fi, (*encFnInfo).kString }
+			case reflect.Float64:
+				fn = encFn{ &fi, (*encFnInfo).kFloat64 }
+			case reflect.Float32:
+				fn = encFn{ &fi, (*encFnInfo).kFloat32 }
+			case reflect.Int, reflect.Int8, reflect.Int64, reflect.Int32, reflect.Int16:
+				fn = encFn{ &fi, (*encFnInfo).kInt }
+			case reflect.Uint8, reflect.Uint64, reflect.Uint, reflect.Uint32, reflect.Uint16:
+				fn = encFn{ &fi, (*encFnInfo).kUint }
+			case reflect.Invalid:
+				fn = encFn{ &fi, (*encFnInfo).kInvalid }
+			case reflect.Slice:
+				fn = encFn{ &fi, (*encFnInfo).kSlice }
+			case reflect.Array:
+				fn = encFn{ &fi, (*encFnInfo).kArray }
+			case reflect.Struct:
+				fn = encFn{ &fi, (*encFnInfo).kStruct }
+			case reflect.Ptr:
+				fn = encFn{ &fi, (*encFnInfo).kPtr }
+			case reflect.Interface:
+				fn = encFn{ &fi, (*encFnInfo).kInterface }
+			case reflect.Map:
+				fn = encFn{ &fi, (*encFnInfo).kMap }
+			default:
+				fn = encFn{ &fi, (*encFnInfo).kErr }
 			}
 		}
-		newlen++
+		e.f[rtid] = fn
 	}
+	
+	fn.f(fn.i, rv)
 
-	if toMap {
-		ee := e.e //don't dereference everytime
-		ee.encodeMapPreamble(newlen)
-		for j := 0; j < newlen; j++ {
-			ee.encodeSymbol(encnames[j])
-			e.encodeValue(rvals[j])
-		}
-	} else {
-		e.e.encodeArrayPreamble(newlen)
-		for j := 0; j < newlen; j++ {
-			e.encodeValue(rvals[j])
-		}
-	}
 }
 
-
 // ----------------------------------------
 
 func (z *ioEncWriter) writeUint16(v uint16) {

codec/ext_dep_test.go

@@ -64,3 +64,11 @@ func Benchmark__VMsgpack_Decode(b *testing.B) {
 func TestMsgpackPythonGenStreams(t *testing.T) {
 	doTestMsgpackPythonGenStreams(t)
 }
+
+func TestMsgpackRpcSpecGoClientToPythonSvc(t *testing.T) {
+	doTestMsgpackRpcSpecGoClientToPythonSvc(t)
+}
+
+func TestMsgpackRpcSpecPythonClientToGoSvc(t *testing.T) {
+	doTestMsgpackRpcSpecPythonClientToGoSvc(t)
+}

codec/helper.go

@@ -7,6 +7,7 @@ package codec
 
 import (
 	"encoding/binary"
+	"errors"
 	"fmt"
 	"reflect"
 	"sort"
@@ -21,9 +22,25 @@ const (
 	// For >= mapAccessThreshold elements, map outways cost of linear search 
 	//   - this was critical for reflect.Type, whose equality cost is pretty high (set to 4)
 	//   - for integers, equality cost is cheap (set to 16, 32 of 64)
-	mapAccessThreshold    = 16 // 4
+	// mapAccessThreshold    = 16 // 4
+	
 	binarySearchThreshold = 16
 	structTagName         = "codec"
+	
+	// Support 
+	//    encoding.BinaryMarshaler: MarshalBinary() (data []byte, err error)
+	//    encoding.BinaryUnmarshaler: UnmarshalBinary(data []byte) error
+	// This constant flag will enable or disable it.
+	// 
+	// Supporting this feature required a map access each time the en/decodeValue 
+	// method is called to get the typeInfo and look at baseId. This caused a 
+	// clear performance degradation. Some refactoring helps a portion of the loss.
+	// 
+	// All the band-aids we can put try to mitigate performance loss due to stack splitting:
+	//    - using smaller functions to reduce func framesize
+	// 
+	// TODO: Look into this again later.
+	supportBinaryMarshal  = true
 )
 
 type charEncoding uint8
@@ -37,12 +54,20 @@ const (
 	c_UTF32BE
 )
 
+type binaryUnmarshaler interface {
+	UnmarshalBinary(data []byte) error
+}
+
+type binaryMarshaler interface {
+	MarshalBinary() (data []byte, err error)
+}
+
 var (
 	bigen               = binary.BigEndian
 	structInfoFieldName = "_struct"
 
-	cachedStructFieldInfos      = make(map[uintptr]*structFieldInfos, 4)
-	cachedStructFieldInfosMutex sync.RWMutex
+	cachedTypeInfo      = make(map[uintptr]*typeInfo, 4)
+	cachedTypeInfoMutex sync.RWMutex
 
 	nilIntfSlice     = []interface{}(nil)
 	intfSliceTyp     = reflect.TypeOf(nilIntfSlice)
@@ -51,13 +76,21 @@ var (
 	ptrByteSliceTyp  = reflect.TypeOf((*[]byte)(nil))
 	mapStringIntfTyp = reflect.TypeOf(map[string]interface{}(nil))
 	mapIntfIntfTyp   = reflect.TypeOf(map[interface{}]interface{}(nil))
+	
 	timeTyp          = reflect.TypeOf(time.Time{})
 	ptrTimeTyp       = reflect.TypeOf((*time.Time)(nil))
 	int64SliceTyp    = reflect.TypeOf([]int64(nil))
-
+	
 	timeTypId        = reflect.ValueOf(timeTyp).Pointer()
+	ptrTimeTypId     = reflect.ValueOf(ptrTimeTyp).Pointer()
 	byteSliceTypId   = reflect.ValueOf(byteSliceTyp).Pointer()
 	
+	binaryMarshalerTyp = reflect.TypeOf((*binaryMarshaler)(nil)).Elem()
+	binaryUnmarshalerTyp = reflect.TypeOf((*binaryUnmarshaler)(nil)).Elem()
+	
+	binaryMarshalerTypId = reflect.ValueOf(binaryMarshalerTyp).Pointer()
+	binaryUnmarshalerTypId = reflect.ValueOf(binaryUnmarshalerTyp).Pointer()
+	
 	intBitsize  uint8 = uint8(reflect.TypeOf(int(0)).Bits())
 	uintBitsize uint8 = uint8(reflect.TypeOf(uint(0)).Bits())
 
@@ -65,24 +98,8 @@ var (
 	bsAll0xff = []byte{0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff}
 )
 
-type encdecHandle struct {
-	encHandle
-	decHandle
-}
-
-func (o *encdecHandle) AddExt(
-	rt reflect.Type,
-	tag byte,
-	encfn func(reflect.Value) ([]byte, error),
-	decfn func(reflect.Value, []byte) error,
-) {
-	rtid := reflect.ValueOf(rt).Pointer()
-	o.addEncodeExt(rtid, tag, encfn)
-	o.addDecodeExt(rtid, rt, tag, decfn)
-}
-
 // Handle is the interface for a specific encoding format.
-//
+// 
 // Typically, a Handle is pre-configured before first time use,
 // and not modified while in use. Such a pre-configured Handle
 // is safe for concurrent access.
@@ -93,45 +110,143 @@ type Handle interface {
 	newDecDriver(r decReader) decDriver
 }
 
+type extTypeTagFn struct {
+	rtid uintptr
+	rt reflect.Type
+	tag byte
+	encFn func(reflect.Value) ([]byte, error)
+	decFn func(reflect.Value, []byte) error
+}
+
+type extHandle map[uintptr]*extTypeTagFn
+
+// AddExt registers an encode and decode function for a reflect.Type.
+// Note that the type must be a named type, and specifically not 
+// a pointer or Interface. An error is returned if that is not honored.
+func (o *extHandle) AddExt(
+	rt reflect.Type,
+	tag byte,
+	encfn func(reflect.Value) ([]byte, error),
+	decfn func(reflect.Value, []byte) error,
+) (err error) {
+	// o is a pointer, because we may need to initialize it
+	if rt.PkgPath() == "" || rt.Kind() == reflect.Interface {
+		err = fmt.Errorf("codec.Handle.AddExt: Takes a named type, especially not a pointer or interface: %T", 
+			reflect.Zero(rt).Interface())
+		return
+	}
+	if o == nil {
+		err = errors.New("codec.Handle.AddExt: Nil (should never happen)")
+		return
+	}
+	rtid := reflect.ValueOf(rt).Pointer()
+	if *o == nil {
+		*o = make(map[uintptr]*extTypeTagFn, 4)
+	}
+	m := *o
+	if encfn == nil || decfn == nil {
+		delete(m, rtid)
+	} else {
+		m[rtid] = &extTypeTagFn { rtid, rt, tag, encfn, decfn }
+	}
+	return
+}
+
+func (o extHandle) getExt(rtid uintptr) *extTypeTagFn {
+	return o[rtid]
+}
+
+func (o extHandle) getExtForTag(tag byte) *extTypeTagFn {
+	for _, v := range o {
+		if v.tag == tag {
+			return v
+		}
+	}
+	return nil
+}
+
+func (o extHandle) getDecodeExtForTag(tag byte) (
+	rv reflect.Value, fn func(reflect.Value, []byte) error) {
+	if x := o.getExtForTag(tag); x != nil {
+		// ext is only registered for base
+		rv = reflect.New(x.rt).Elem()
+		fn = x.decFn
+	}
+	return
+}
+
+func (o extHandle) getDecodeExt(rtid uintptr) (tag byte, fn func(reflect.Value, []byte) error) {
+	if x, ok := o[rtid]; ok {
+		tag = x.tag
+		fn = x.decFn
+	}
+	return
+}
+
+func (o extHandle) getEncodeExt(rtid uintptr) (tag byte, fn func(reflect.Value) ([]byte, error)) {
+	if x, ok := o[rtid]; ok {
+		tag = x.tag
+		fn = x.encFn
+	}
+	return
+}
+
+// typeInfo keeps information about each type referenced in the encode/decode sequence.
+// 
+// During an encode/decode sequence, we work as below:
+//   - If base is a built in type, en/decode base value
+//   - If base is registered as an extension, en/decode base value
+//   - If type is binary(M/Unm)arshaler, call Binary(M/Unm)arshal method
+//   - Else decode appropriately based on the reflect.Kind
+type typeInfo struct {
+	sis       []*structFieldInfo // sorted. Used when enc/dec struct to map.
+	sisp      []*structFieldInfo // unsorted. Used when enc/dec struct to array.
+	// base      reflect.Type
+	
+	// baseId is the pointer to the base reflect.Type, after deferencing
+	// the pointers. E.g. base type of ***time.Time is time.Time.
+	baseId    uintptr
+	baseIndir int8 // number of indirections to get to base
+	m         bool // base type (T or *T) is a binaryMarshaler
+	unm       bool // base type (T or *T) is a binaryUnmarshaler
+	mIndir    int8 // number of indirections to get to binaryMarshaler type
+	unmIndir  int8 // number of indirections to get to binaryUnmarshaler type
+	toArray   bool // whether this (struct) type should be encoded as an array
+}
+
 type structFieldInfo struct {
 	encName   string // encode name
-	is        []int
+	
+	// only one of 'i' or 'is' can be set. If 'i' is -1, then 'is' has been set.
+	
+	is        []int // (recursive/embedded) field index in struct
 	i         int16 // field index in struct
-	omitEmpty bool
-	toArray   bool
+	omitEmpty bool  
+	toArray   bool  // if field is _struct, is the toArray set?
+	
 	// tag       string   // tag
 	// name      string   // field name
 	// encNameBs []byte   // encoded name as byte stream
 	// ikind     int      // kind of the field as an int i.e. int(reflect.Kind)
 }
 
-type structFieldInfos struct {
-	sis     []structFieldInfo // sorted. Used when enc/dec struct to map.
-	sisp    []structFieldInfo // unsorted. Used when enc/dec struct to array.
-	toArray bool 
-}
-
 type sfiSortedByEncName []*structFieldInfo
 
-func (p sfiSortedByEncName) Len() int {
-	return len(p)
-}
-
-func (p sfiSortedByEncName) Less(i, j int) bool {
-	return p[i].encName < p[j].encName
-}
-
-func (p sfiSortedByEncName) Swap(i, j int) {
-	p[i], p[j] = p[j], p[i]
-}
+func (p sfiSortedByEncName) Len() int           { return len(p) }
+func (p sfiSortedByEncName) Less(i, j int) bool { return p[i].encName < p[j].encName }
+func (p sfiSortedByEncName) Swap(i, j int)      { p[i], p[j] = p[j], p[i] }
 
-func (sis *structFieldInfos) indexForEncName(name string) int {
-	sissis := sis.sis 
-	sislen := len(sissis)
-	if sislen < binarySearchThreshold {
+func (sis *typeInfo) indexForEncName(name string) int {
+	//sissis := sis.sis 
+	if sislen := len(sis.sis); sislen < binarySearchThreshold {
 		// linear search. faster than binary search in my testing up to 16-field structs.
-		for i := 0; i < sislen; i++ {
-			if sissis[i].encName == name {
+		// for i := 0; i < sislen; i++ {
+		// 	if sis.sis[i].encName == name {
+		// 		return i
+		// 	}
+		// }
+		for i, si := range sis.sis {
+			if si.encName == name {
 				return i
 			}
 		}
@@ -141,58 +256,89 @@ func (sis *structFieldInfos) indexForEncName(name string) int {
 		for i < j {
 			h = i + (j-i)/2
 			// i ≤ h < j
-			if sissis[h].encName < name {
+			if sis.sis[h].encName < name {
 				i = h + 1 // preserves f(i-1) == false
 			} else {
 				j = h // preserves f(j) == true
 			}
 		}
-		if i < sislen && sissis[i].encName == name {
+		if i < sislen && sis.sis[i].encName == name {
 			return i
 		}
 	}
 	return -1
 }
 
-func getStructFieldInfos(rtid uintptr, rt reflect.Type) (sis *structFieldInfos) {
+func getTypeInfo(rtid uintptr, rt reflect.Type) (sis *typeInfo) {
 	var ok bool
-	cachedStructFieldInfosMutex.RLock()
-	sis, ok = cachedStructFieldInfos[rtid]
-	cachedStructFieldInfosMutex.RUnlock()
+	cachedTypeInfoMutex.RLock()
+	sis, ok = cachedTypeInfo[rtid]
+	cachedTypeInfoMutex.RUnlock()
 	if ok {
 		return
 	}
 
-	cachedStructFieldInfosMutex.Lock()
-	defer cachedStructFieldInfosMutex.Unlock()
-	if sis, ok = cachedStructFieldInfos[rtid]; ok {
+	cachedTypeInfoMutex.Lock()
+	defer cachedTypeInfoMutex.Unlock()
+	if sis, ok = cachedTypeInfo[rtid]; ok {
 		return
 	}
 
-	sis = new(structFieldInfos)
-	var siInfo *structFieldInfo
-	if f, ok := rt.FieldByName(structInfoFieldName); ok {
-		siInfo = parseStructFieldInfo(structInfoFieldName, f.Tag.Get(structTagName))
-		sis.toArray = siInfo.toArray
+	sis = new(typeInfo)
+	
+	var indir int8
+	if ok, indir = implementsIntf(rt, binaryMarshalerTyp); ok {
+		sis.m, sis.mIndir = true, indir
 	}
-	sisp := make([]*structFieldInfo, 0, rt.NumField())
-	rgetStructFieldInfos(rt, nil, make(map[string]bool), &sisp, siInfo)
-	sis.sisp = make([]structFieldInfo, len(sisp))
-	sis.sis = make([]structFieldInfo, len(sisp))
-	for i := 0; i < len(sisp); i++ {
-		sis.sisp[i] = *sisp[i]
+	if ok, indir = implementsIntf(rt, binaryUnmarshalerTyp); ok {
+		sis.unm, sis.unmIndir = true, indir
 	}
 	
-	sort.Sort(sfiSortedByEncName(sisp))
-	for i := 0; i < len(sisp); i++ {
-		sis.sis[i] = *sisp[i]
+	pt := rt
+	var ptIndir int8 
+	for ; pt.Kind() == reflect.Ptr; pt, ptIndir = pt.Elem(), ptIndir+1 { }
+	if ptIndir == 0 {
+		//sis.base = rt 
+		sis.baseId = rtid
+	} else {
+		//sis.base = pt 
+		sis.baseId = reflect.ValueOf(pt).Pointer()
+		sis.baseIndir = ptIndir
+	}
+	
+	if rt.Kind() == reflect.Struct {
+		var siInfo *structFieldInfo
+		if f, ok := rt.FieldByName(structInfoFieldName); ok {
+			siInfo = parseStructFieldInfo(structInfoFieldName, f.Tag.Get(structTagName))
+			sis.toArray = siInfo.toArray
+		}
+		sisp := make([]*structFieldInfo, 0, rt.NumField())
+		rgetTypeInfo(rt, nil, make(map[string]bool), &sisp, siInfo)
+
+		// // try to put all si close together
+		// const tryToPutAllStructFieldInfoTogether = true
+		// if tryToPutAllStructFieldInfoTogether {
+		// 	sisp2 := make([]structFieldInfo, len(sisp))
+		// 	for i, si := range sisp {
+		// 		sisp2[i] = *si
+		// 	}
+		// 	for i := range sisp {
+		// 		sisp[i] = &sisp2[i]
+		// 	}
+		// }
+		
+		sis.sisp = make([]*structFieldInfo, len(sisp))
+		sis.sis = make([]*structFieldInfo, len(sisp))
+		copy(sis.sisp, sisp)
+		sort.Sort(sfiSortedByEncName(sisp))
+		copy(sis.sis, sisp)
 	}
 	// sis = sisp
-	cachedStructFieldInfos[rtid] = sis
+	cachedTypeInfo[rtid] = sis
 	return
 }
 
-func rgetStructFieldInfos(rt reflect.Type, indexstack []int, fnameToHastag map[string]bool,
+func rgetTypeInfo(rt reflect.Type, indexstack []int, fnameToHastag map[string]bool,
 	sis *[]*structFieldInfo, siInfo *structFieldInfo,
 ) {
 	for j := 0; j < rt.NumField(); j++ {
@@ -208,12 +354,13 @@ func rgetStructFieldInfos(rt reflect.Type, indexstack []int, fnameToHastag map[s
 			//if anonymous, inline it if there is no struct tag, else treat as regular field
 			if stag == "" {
 				indexstack2 := append(append([]int(nil), indexstack...), j)
-				rgetStructFieldInfos(f.Type, indexstack2, fnameToHastag, sis, siInfo)
+				rgetTypeInfo(f.Type, indexstack2, fnameToHastag, sis, siInfo)
 				continue
 			}
 		}
-		//do not let fields with same name in embedded structs override field at higher level.
-		//this must be done after anonymous check, to allow anonymous field still include their child fields
+		// do not let fields with same name in embedded structs override field at higher level.
+		// this must be done after anonymous check, to allow anonymous field 
+		// still include their child fields
 		if _, ok := fnameToHastag[f.Name]; ok {
 			continue
 		}
@@ -280,25 +427,3 @@ func doPanic(tag string, format string, params ...interface{}) {
 	panic(fmt.Errorf("%s: "+format, params2...))
 }
 
-//--------------------------------------------------
-
-// // This implements the util.Codec interface
-// type Codec struct {
-// 	H Handle
-// }
-
-// func (x Codec) Encode(w io.Writer, v interface{}) error {
-// 	return NewEncoder(w, x.H).Encode(v)
-// }
-
-// func (x Codec) EncodeBytes(out *[]byte, v interface{}) error {
-// 	return NewEncoderBytes(out, x.H).Encode(v)
-// }
-
-// func (x Codec) Decode(r io.Reader, v interface{}) error {
-// 	return NewDecoder(r, x.H).Decode(v)
-// }
-
-// func (x Codec) DecodeBytes(in []byte, v interface{}) error {
-// 	return NewDecoderBytes(in, x.H).Decode(v)
-// }

codec/helper_internal.go

@@ -9,6 +9,7 @@ package codec
 import (
 	"errors"
 	"fmt"
+	"math"
 	"reflect"
 )
 
@@ -82,3 +83,34 @@ func pruneSignExt(v []byte) (n int) {
 	}
 	return
 }
+
+func implementsIntf(typ, iTyp reflect.Type) (success bool, indir int8) {
+	if typ == nil {
+		return
+	}
+	rt := typ
+	// The type might be a pointer and we need to keep
+	// dereferencing to the base type until we find an implementation.
+	for {
+		if rt.Implements(iTyp) {
+			return true, indir
+		}
+		if p := rt; p.Kind() == reflect.Ptr {
+			indir++
+			if indir >= math.MaxInt8 { // insane number of indirections
+				return false, 0
+			}
+			rt = p.Elem()
+			continue
+		}
+		break
+	}
+	// No luck yet, but if this is a base type (non-pointer), the pointer might satisfy.
+	if typ.Kind() != reflect.Ptr {
+		// Not a pointer, but does the pointer work?
+		if reflect.PtrTo(typ).Implements(iTyp) {
+			return true, -1
+		}
+	}
+	return false, 0
+}

codec/msgpack.go

@@ -68,6 +68,14 @@ const (
 // the msgpack spec at http://wiki.msgpack.org/display/MSGPACK/RPC+specification
 var MsgpackSpecRpc msgpackSpecRpc
 
+// MsgpackSpecRpcMultiArgs is a special type which signifies to the MsgpackSpecRpcCodec
+// that the backend RPC service takes multiple arguments, which have been arranged 
+// in sequence in the slice. 
+// 
+// The Codec then passes it AS-IS to the rpc service (without wrapping it in an 
+// array of 1 element).
+type MsgpackSpecRpcMultiArgs []interface{}
+
 // A MsgpackContainer type specifies the different types of msgpackContainers.
 type msgpackContainerType struct {
 	fixCutoff             int
@@ -106,7 +114,7 @@ type MsgpackHandle struct {
 	// a []byte or string based on the setting of RawToString.
 	WriteExt bool
 
-	encdecHandle
+	extHandle
 	EncodeOptions
 	DecodeOptions
 }
@@ -124,10 +132,12 @@ type msgpackDecDriver struct {
 	bdType decodeEncodedType
 }
 
-func (e *msgpackEncDriver) encodeBuiltinType(rt uintptr, rv reflect.Value) bool {
+func (e *msgpackEncDriver) isBuiltinType(rt uintptr) bool {
 	//no builtin types. All encodings are based on kinds. Types supported as extensions.
 	return false
 }
+	
+func (e *msgpackEncDriver) encodeBuiltinType(rt uintptr, rv reflect.Value) {}
 
 func (e *msgpackEncDriver) encodeNil() {
 	e.w.writen1(mpNil)
@@ -267,9 +277,12 @@ func (e *msgpackEncDriver) writeContainerLen(ct msgpackContainerType, l int) {
 
 //---------------------------------------------
 
-func (d *msgpackDecDriver) decodeBuiltinType(rt uintptr, rv reflect.Value) bool {
+func (d *msgpackDecDriver) isBuiltinType(rt uintptr) bool {
+	//no builtin types. All encodings are based on kinds. Types supported as extensions.
 	return false
 }
+	
+func (d *msgpackDecDriver) decodeBuiltinType(rt uintptr, rv reflect.Value) {}
 
 // Note: This returns either a primitive (int, bool, etc) for non-containers,
 // or a containerType, or a specific type denoting nil or extension.
@@ -329,11 +342,15 @@ func (d *msgpackDecDriver) decodeNaked() (rv reflect.Value, ctx decodeNakedConte
 				var rvm string
 				rv = reflect.ValueOf(&rvm).Elem()
 			} else {
-				rv = reflect.New(byteSliceTyp).Elem() // Use New, not Zero, so it's settable
+				rvm := []byte{}
+				rv = reflect.ValueOf(&rvm).Elem()
+				//rv = reflect.New(byteSliceTyp).Elem() // Use New, not Zero, so it's settable
 			}
 		case bd == mpBin8, bd == mpBin16, bd == mpBin32:
 			ctx = dncContainer
-			rv = reflect.New(byteSliceTyp).Elem()
+			rvm := []byte{}
+			rv = reflect.ValueOf(&rvm).Elem()
+			// rv = reflect.New(byteSliceTyp).Elem()
 		case bd == mpArray16, bd == mpArray32, bd >= mpFixArrayMin && bd <= mpFixArrayMax:
 			ctx = dncContainer
 			// v = containerList
@@ -684,32 +701,58 @@ func (d *msgpackDecDriver) decodeExt(tag byte) (xbs []byte) {
 
 //--------------------------------------------------
 
-func (msgpackSpecRpc) ServerCodec(conn io.ReadWriteCloser, h Handle) rpc.ServerCodec {
+func (x msgpackSpecRpc) ServerCodec(conn io.ReadWriteCloser, h Handle) rpc.ServerCodec {
 	return &msgpackSpecRpcCodec{newRPCCodec(conn, h)}
 }
 
-func (msgpackSpecRpc) ClientCodec(conn io.ReadWriteCloser, h Handle) rpc.ClientCodec {
+func (x msgpackSpecRpc) ClientCodec(conn io.ReadWriteCloser, h Handle) rpc.ClientCodec {
 	return &msgpackSpecRpcCodec{newRPCCodec(conn, h)}
 }
 
 // /////////////// Spec RPC Codec ///////////////////
-func (c msgpackSpecRpcCodec) WriteRequest(r *rpc.Request, body interface{}) error {
-	return c.writeCustomBody(0, r.Seq, r.ServiceMethod, body)
+func (c *msgpackSpecRpcCodec) WriteRequest(r *rpc.Request, body interface{}) error {
+	// WriteRequest can write to both a Go service, and other services that do 
+	// not abide by the 1 argument rule of a Go service. 
+	// We discriminate based on if the body is a MsgpackSpecRpcMultiArgs
+	var bodyArr []interface{}
+	if m, ok := body.(MsgpackSpecRpcMultiArgs); ok {
+		bodyArr = ([]interface{})(m)
+	} else {
+		bodyArr = []interface{}{body}
+	}
+	r2 := []interface{}{ 0, uint32(r.Seq), r.ServiceMethod, bodyArr }
+	return c.write(r2, nil, false, true)
 }
 
-func (c msgpackSpecRpcCodec) WriteResponse(r *rpc.Response, body interface{}) error {
-	return c.writeCustomBody(1, r.Seq, r.Error, body)
+func (c *msgpackSpecRpcCodec) WriteResponse(r *rpc.Response, body interface{}) error {
+	var moe interface{}
+	if r.Error != "" {
+		moe = r.Error
+	}
+	if moe != nil && body != nil {
+		body = nil
+	}
+	r2 := []interface{}{ 1, uint32(r.Seq), moe, body }
+	return c.write(r2, nil, false, true)
 }
 
-func (c msgpackSpecRpcCodec) ReadResponseHeader(r *rpc.Response) error {
+func (c *msgpackSpecRpcCodec) ReadResponseHeader(r *rpc.Response) error {
 	return c.parseCustomHeader(1, &r.Seq, &r.Error)
 }
 
-func (c msgpackSpecRpcCodec) ReadRequestHeader(r *rpc.Request) error {
+func (c *msgpackSpecRpcCodec) ReadRequestHeader(r *rpc.Request) error {
 	return c.parseCustomHeader(0, &r.Seq, &r.ServiceMethod)
 }
 
-func (c msgpackSpecRpcCodec) parseCustomHeader(expectTypeByte byte, msgid *uint64, methodOrError *string) (err error) {
+func (c *msgpackSpecRpcCodec) ReadRequestBody(body interface{}) error {
+	if body == nil { // read and discard
+		return c.read(nil)
+	}
+	bodyArr := []interface{}{body}
+	return c.read(&bodyArr)
+}
+
+func (c *msgpackSpecRpcCodec) parseCustomHeader(expectTypeByte byte, msgid *uint64, methodOrError *string) (err error) {
 
 	// We read the response header by hand
 	// so that the body can be decoded on its own from the stream at a later time.
@@ -745,43 +788,18 @@ func (c msgpackSpecRpcCodec) parseCustomHeader(expectTypeByte byte, msgid *uint6
 	return
 }
 
-func (c msgpackSpecRpcCodec) writeCustomBody(typeByte byte, msgid uint64, methodOrError string, body interface{}) (err error) {
-	var moe interface{} = methodOrError
-	// response needs nil error (not ""), and only one of error or body can be nil
-	if typeByte == 1 {
-		if methodOrError == "" {
-			moe = nil
-		}
-		if moe != nil && body != nil {
-			body = nil
-		}
-	}
-	r2 := []interface{}{typeByte, uint32(msgid), moe, body}
-	return c.write(r2, nil, false, true)
-}
-
 //--------------------------------------------------
 
-// BinaryEncodeExt returns the underlying bytes of this value AS-IS.
-// Configure this to support the Binary Extension using tag 0.
-func (_ *MsgpackHandle) BinaryEncodeExt(rv reflect.Value) ([]byte, error) {
-	if rv.IsNil() {
-		return nil, nil
-	}
-	return rv.Bytes(), nil
-}
-
-// BinaryDecodeExt sets passed byte slice AS-IS into the reflect Value.
-// Configure this to support the Binary Extension using tag 0.
-func (_ *MsgpackHandle) BinaryDecodeExt(rv reflect.Value, bs []byte) (err error) {
-	rv.SetBytes(bs)
-	return
-}
-
 // TimeEncodeExt encodes a time.Time as a byte slice.
 // Configure this to support the Time Extension, e.g. using tag 1.
 func (_ *MsgpackHandle) TimeEncodeExt(rv reflect.Value) (bs []byte, err error) {
-	bs = encodeTime(rv.Interface().(time.Time))
+	rvi := rv.Interface()
+	switch iv := rvi.(type) {
+	case time.Time:
+		bs = encodeTime(iv)
+	default:
+		err = fmt.Errorf("codec/msgpack: TimeEncodeExt expects a time.Time. Received %T", rvi)
+	}
 	return
 }
 

codec/msgpack_test.py

@@ -4,7 +4,7 @@
 # A Test calls this internally to create the golden files
 # So it can process them (so we don't have to checkin the files).
 
-import msgpack, sys, os
+import msgpack, msgpackrpc, sys, os, threading
 
 def get_test_data_list():
     # get list with all primitive types, and a combo type
@@ -61,11 +61,50 @@ def build_test_data(destdir):
         f.write(serialized)
         f.close()
 
+def doRpcServer(port, stopTimeSec):
+    class EchoHandler(object):
+        def Echo123(self, msg1, msg2, msg3):
+            return ("1:%s 2:%s 3:%s" % (msg1, msg2, msg3))
+        def EchoStruct(self, msg):
+            return ("%s" % msg)
+    
+    addr = msgpackrpc.Address('localhost', port)
+    server = msgpackrpc.Server(EchoHandler())
+    server.listen(addr)
+    # run thread to stop it after stopTimeSec seconds if > 0
+    if stopTimeSec > 0:
+        def myStopRpcServer():
+            server.stop()
+        t = threading.Timer(stopTimeSec, myStopRpcServer)
+        t.start()
+    server.start()
+
+def doRpcClientToPythonSvc(port):
+    address = msgpackrpc.Address('localhost', port)
+    client = msgpackrpc.Client(address, unpack_encoding='utf-8')
+    print client.call("Echo123", "A1", "B2", "C3")
+    print client.call("EchoStruct", {"A" :"Aa", "B":"Bb", "C":"Cc"})
+   
+def doRpcClientToGoSvc(port):
+    # print ">>>> port: ", port, " <<<<<"
+    address = msgpackrpc.Address('localhost', port)
+    client = msgpackrpc.Client(address, unpack_encoding='utf-8')
+    print client.call("TestRpcInt.Echo123", ["A1", "B2", "C3"])
+    print client.call("TestRpcInt.EchoStruct", {"A" :"Aa", "B":"Bb", "C":"Cc"})
+
 def doMain(args):
     if len(args) == 2 and args[0] == "testdata":
         build_test_data(args[1])
+    elif len(args) == 3 and args[0] == "rpc-server":
+        doRpcServer(int(args[1]), int(args[2]))
+    elif len(args) == 2 and args[0] == "rpc-client-python-service":
+        doRpcClientToPythonSvc(int(args[1]))
+    elif len(args) == 2 and args[0] == "rpc-client-go-service":
+        doRpcClientToGoSvc(int(args[1]))
     else:
-        print("Usage: build.py [testdata]")
+        print("Usage: msgpack_test.py " + 
+              "[testdata|rpc-server|rpc-client-python-service|rpc-client-go-service] ...")
     
 if __name__ == "__main__":
     doMain(sys.argv[1:])
+

codec/rpc.go

@@ -40,11 +40,11 @@ type goRpcCodec struct {
 type goRpc struct{}
 
 func (x goRpc) ServerCodec(conn io.ReadWriteCloser, h Handle) rpc.ServerCodec {
-	return goRpcCodec{newRPCCodec(conn, h)}
+	return &goRpcCodec{newRPCCodec(conn, h)}
 }
 
 func (x goRpc) ClientCodec(conn io.ReadWriteCloser, h Handle) rpc.ClientCodec {
-	return goRpcCodec{newRPCCodec(conn, h)}
+	return &goRpcCodec{newRPCCodec(conn, h)}
 }
 
 func newRPCCodec(conn io.ReadWriteCloser, h Handle) rpcCodec {
@@ -60,7 +60,7 @@ func newRPCCodec(conn io.ReadWriteCloser, h Handle) rpcCodec {
 }
 
 // /////////////// RPC Codec Shared Methods ///////////////////
-func (c rpcCodec) write(obj1, obj2 interface{}, writeObj2, doFlush bool) (err error) {
+func (c *rpcCodec) write(obj1, obj2 interface{}, writeObj2, doFlush bool) (err error) {
 	if err = c.enc.Encode(obj1); err != nil {
 		return
 	}
@@ -77,7 +77,7 @@ func (c rpcCodec) write(obj1, obj2 interface{}, writeObj2, doFlush bool) (err er
 }
 
 
-func (c rpcCodec) read(obj interface{}) (err error) {
+func (c *rpcCodec) read(obj interface{}) (err error) {
 	//If nil is passed in, we should still attempt to read content to nowhere.
 	if obj == nil {
 		var obj2 interface{}
@@ -86,31 +86,31 @@ func (c rpcCodec) read(obj interface{}) (err error) {
 	return c.dec.Decode(obj)
 }
 
-func (c rpcCodec) Close() error {
+func (c *rpcCodec) Close() error {
 	return c.rwc.Close()
 }
 
-func (c rpcCodec) ReadResponseBody(body interface{}) error {
+func (c *rpcCodec) ReadResponseBody(body interface{}) error {
 	return c.read(body)
 }
 
-func (c rpcCodec) ReadRequestBody(body interface{}) error {
+func (c *rpcCodec) ReadRequestBody(body interface{}) error {
 	return c.read(body)
 }
 
 // /////////////// Go RPC Codec ///////////////////
-func (c goRpcCodec) WriteRequest(r *rpc.Request, body interface{}) error {
+func (c *goRpcCodec) WriteRequest(r *rpc.Request, body interface{}) error {
 	return c.write(r, body, true, true)
 }
 
-func (c goRpcCodec) WriteResponse(r *rpc.Response, body interface{}) error {
+func (c *goRpcCodec) WriteResponse(r *rpc.Response, body interface{}) error {
 	return c.write(r, body, true, true)
 }
 
-func (c goRpcCodec) ReadResponseHeader(r *rpc.Response) error {
+func (c *goRpcCodec) ReadResponseHeader(r *rpc.Response) error {
 	return c.read(r)
 }
 
-func (c goRpcCodec) ReadRequestHeader(r *rpc.Request) error {
+func (c *goRpcCodec) ReadRequestHeader(r *rpc.Request) error {
 	return c.read(r)
 }

codec/time.go

@@ -104,19 +104,36 @@ func decodeTime(bs []byte) (tt time.Time, err error) {
 	i2 = i + 2
 	tz = bigen.Uint16(bs[i:i2])
 	i = i2
-	var tzname = []byte("UTC+00:00")
 	// sign extend sign bit into top 2 MSB (which were dst bits):
 	if tz&(1<<13) == 0 { // positive
 		tz = tz & 0x3fff //clear 2 MSBs: dst bits
 	} else { // negative
 		tz = tz | 0xc000 //set 2 MSBs: dst bits
-		tzname[3] = '-'
+		//tzname[3] = '-' (TODO: verify. this works here)
+	}
+	tzint := int16(tz)
+	if tzint == 0 {
+		tt = time.Unix(tsec, int64(tnsec)).UTC()
+	} else {
+		// For Go Time, do not use a descriptive timezone. 
+		// It's unnecessary, and makes it harder to do a reflect.DeepEqual.
+		// The Offset already tells what the offset should be, if not on UTC and unknown zone name.
+		// var zoneName = timeLocUTCName(tzint)
+		tt = time.Unix(tsec, int64(tnsec)).In(time.FixedZone("", int(tzint)*60))
+	}
+	return
+}
+
+func timeLocUTCName(tzint int16) string {
+	if tzint == 0 {
+		return "UTC"
 	}
-	tzint := (int16(tz))
+	var tzname = []byte("UTC+00:00")
 	//tzname := fmt.Sprintf("UTC%s%02d:%02d", tzsign, tz/60, tz%60) //perf issue using Sprintf. inline below.
 	//tzhr, tzmin := tz/60, tz%60 //faster if u convert to int first
 	var tzhr, tzmin int16
 	if tzint < 0 {
+		tzname[3] = '-' // (TODO: verify. this works here)
 		tzhr, tzmin = -tzint/60, (-tzint)%60
 	} else {
 		tzhr, tzmin = tzint/60, tzint%60
@@ -125,7 +142,10 @@ func decodeTime(bs []byte) (tt time.Time, err error) {
 	tzname[5] = timeDigits[tzhr%10]
 	tzname[7] = timeDigits[tzmin/10]
 	tzname[8] = timeDigits[tzmin%10]
-
-	tt = time.Unix(tsec, int64(tnsec)).In(time.FixedZone(string(tzname), int(tzint)*60))
-	return
+	return string(tzname)
+	//return time.FixedZone(string(tzname), int(tzint)*60)
 }
+
+
+
+