RpcLibrary

 

This library wraps PInvoke calls to the Win32 RPC APIs. This allows pure managed C# applications the ability to move byte arrays over the supported rpc protocols. The following is a basic demonstration:

void Test()
{
    //An id to identify the endpoint interface
    Guid iid = Guid.NewGuid();
    using (RpcServerApi server = new RpcServerApi(iid))
    {
        //Allow up to 5 connections over named pipes
        server.AddProtocol(RpcProtseq.ncacn_np, @"\pipe\testpipename", 5);
        //Authenticate via WinNT
        server.AddAuthentication(RpcAuthentication.RPC_C_AUTHN_WINNT);
        //Start receiving calls
        server.StartListening();
        //When a call comes, do the following:
        server.OnExecute +=
            delegate(IRpcClientInfo client, byte[] arg)
            {
                using (client.Impersonate())
                    return new byte[0];
            };

        //For the client, we specify the protocol, endpoint, and interface id to connect
        using (RpcClientApi client = new RpcClientApi(iid, RpcProtseq.ncacn_np, null, @"\pipe\testpipename"))
        {
            //Authenticate as ourselves
            client.AuthenticateAs(RpcClientApi.Self);
            //Call the server
            byte[] response = client.Execute(new byte[0]);
        }
    }
}

Protocols tested include named-pipes, LRPC, and TCP/IP. I’m sure you could get others working as well, but this was enough for a good start.

So someone might start asking “But why would you do this, why not use WCF, TCP Listener, Http Listener, etc?” I’ll try to address these, but you really need to have former experience with RPC to truely understand the benefits…

  • WCF and .Net Remoting are dog slow by comparison, up to 20 times slower.
  • WCF (IMHO) is poorly designed, have you seen anyone provide their own serialization or transport? Nope you havn’t. The original .Net Remoting layer did a much better job in the design; however, it did not provide an out-of-the-box transport layer that was even remotely acceptable.
  • Try it for yourself: A simple test of a WCF server and 10 client processes with 10 threads will fail horribly once the message complexity grows to a reasonable amount (around 100 FileInfo structures should suffice to blow it up).
  • WCF has it’s place, it’s easy, has a wider platform support, but it’s just not very good in reliability and it’s throughput and parallel processing capabilities are far from ideal.
  • Custom listeners (Tcp/NetPipe/Http) can also be useful; however, they simply cannot provide the robust channel that RPC will provide over the same protocols.

The next objection that seems obvious enough: your left with only a transport. There is no ability to serialize ‘messages’ from client to server and back. This is by choice, the transport provided here is intended to be only that, a transport. It moves bytes from one process to another, that is all it does. I’m currently working on final touches to a proto-buffer wrapper around this transport using Jon Skeet’s dotnet-protobufs. It’s fast, reliable, secure, and extensible… all the things I could only wish Microsoft had delivered in .Net 1.0.

Links
 

I ran across another post of someone looking to get rid of WCF on StackOverflow today. The post titled “WCF replacement for cross process/machine communication” goes into the typical complaints about configuration of WCF. I actually think this is the least of the issues I’ve had with WCF. Whatever your reason for looking to abandon WCF, this post is for you. A step-by-step walk-through to get up and running with protobuffers over Win32 rpc.

Step 1 – Gathering dependencies

For this post I’m going to be using VStudio 2008. The primary reason is to show the explicit use of NuGet rather than depending on Visual Studio to do it for us. Now let’s get started. Start by creating a new project in Visual Studio, for this I’m going to use a simple command-line application named “SampleProtoRpc”.

After you have created the project, right-click the project and select “New Folder” and type the name “Depends”. Now visit the NuGet project page and download the “NuGet.exe Command Line bootstrapper”. It should be a single file, “NuGet.exe”. Place this file in the newly created “Depends” directory. From a command-prompt, run NuGet.exe to ensure that you are up and running.

Now right-click the project and select the “Properites” from the bottom. In the properties window, click the “Build Events” tab on the left. In the “Pre-build event command line:” text box enter the following text:

"$(ProjectDir)Depends\NuGet.exe" INSTALL Google.ProtocolBuffers.Rpc -OutputDirectory "$(ProjectDir)Depends" -ExcludeVersion -Version 1.11.1016.3

You can update the version to the latest by checking the current version at http://nuget.org/packages/Google.ProtocolBuffers.Rpc. The reason to use a fixed version is to prevent NuGet from constantly checking with the server to see if it has the latest version. By pinning the version number NuGet.exe will make a quick check and continue if it exists.

Click “Build” and view the Output window, it should contain something like the following text:

------ Build started: Project: SampleProtoRpc, Configuration: Debug Any CPU ------
"E:\Projects\Templates\SampleProtoRpc\Depends\NuGet.exe" INSTALL Google.ProtocolBuffers.Rpc -OutputDirectory "E:\Projects\Templates\SampleProtoRpc\Depends" -ExcludeVersion -Version 1.11.1016.3
Attempting to resolve dependency 'CSharpTest.Net.RpcLibrary (≥ 1.11.924.348)'.
Attempting to resolve dependency 'Google.ProtocolBuffers (≥ 2.4.1.473)'.
Successfully installed 'CSharpTest.Net.RpcLibrary 1.11.924.348'.
Successfully installed 'Google.ProtocolBuffers 2.4.1.473'.
Successfully installed 'Google.ProtocolBuffers.Rpc 1.11.1016.3'.
SampleProtoRpc -> E:\Projects\Templates\SampleProtoRpc\bin\Debug\SampleProtoRpc.exe
========== Build: 1 succeeded or up-to-date, 0 failed, 0 skipped ==========

Lastly we need to add the dependencies to the project. Right-click the “References” folder in the project and select “Add References…”. Click on the “Browse” tab in the resulting “Add Reference” dialog box. For each of the following files navigate to the directory and select the file:

  • Depends\Google.ProtocolBuffers\lib\net20\Google.ProtocolBuffers.dll
  • Depends\CSharpTest.Net.RpcLibrary\lib\net20\CSharpTest.Net.RpcLibrary.dll
  • Depends\Google.ProtocolBuffers.Rpc\lib\net20\Google.ProtocolBuffers.Rpc.dll

Don’t worry about these being ‘net20′ assemblies, it will work fine in 3.5. If you insist upon native 4.0 images, the first two packages contain net40 versions; however, the Google.ProtocolBuffers.Rpc does not at this time. You will need to pull the source and build a 4.0 version for that library.

Step 2 – Defining a Protocol

Now that we have a project containing the correct dependencies we need to add a protocol definition file. This is a very easy format to write in, if you need help see the Google Protocol Buffers Language Guide. For now let’s get started by right-clicking the project, and selecting “Add” -> “New Item…”. Select the “General” tab on the left, and then pick the “Text File” option from the right-hand list. In the “Name” field, enter “Sample.proto” and click the “Add” button.

Once you have created the file, select “File” -> “Save As…” from the menu. Next to the “Save” button click on the drop-down arrow and choose “Save with Encoding”. Answer “Yes” when prompted to overwrite the file. From the “Encoding:” list, choose the option “US-ASCII – Codepage 20127″ near the end of the list and then click “OK”.

Now we can type a protocol buffer definition in this file. For now we are going to use the following:

package Sample;
option optimize_for = SPEED;

message MyRequest {
  required string message = 1;
}

message MyResponse {
}

service MyService {
  rpc Send (MyRequest) returns (MyResponse);
}

And this will be our first service. To generate the source, we are going to add another pre-build event. Right-click the project and select “Properties” again. On the “Build Events” pane add the following line:

"$(ProjectDir)Depends\Google.ProtocolBuffers\tools\ProtoGen.exe" --proto_path="$(ProjectDir)\" -output_directory="$(ProjectDir)\" -cls_compliance=false -service_generator_type=IRPCDISPATCH "$(ProjectDir)Sample.proto"

You should now be able to build the project successfully. Once completed, right-click the project and choose “Add” -> “Existing Item…”, then select the “Sample.cs” that should appear next to the “Sample.proto” file we created. Admittingly this is a crufty integration at best.

If you are not opposed to it, I would recommend using CmdTool.exe with a configuration similar to this example. CmdTool.exe is available in one of these downloads, you just download, run “CmdTool.exe Register” and save the configuration example in the same directory as the project. That’s about it, you now have all the source generated to do the work.

Step 3 – Implementing the Service

Before we can go much further we must implement our service interface. Right-click the project and add a new Class file so we can create our implementation, I just called mine “Implementation”. The interface has already been defined for us, it’s name will be our service’s name prepended with an ‘I’. Here is a first-pass implementation that simply writes the message to the console.

    class Implementation : IMyService
    {
        #region IMyService Members
        public MyResponse Send(MyRequest myRequest)
        {
            using (WindowsIdentity user = WindowsIdentity.GetCurrent())
            {
                Console.WriteLine("{0} says: {1}", user.Name, myRequest.Message);
                return MyResponse.DefaultInstance;
            }
        }
        #endregion
    }

Step 4 – Setting up the Listener

Time to start playing with our server-side listener. For this example we are going to allow lrpc, tcp, or named pipes. The static IID defines the interface we want to talk to. We can host several interfaces from this process, but this example will only use one. You will notice we create the generated server proxy “MyService.ServerStub” by handing it an implementation of the IMyService interface. This server stub can then be used to create the RpcServer instance. Once we add at least one protocol and call StartListening we are ready to receive calls. The setup is trivial, so without further explanation here is our new ‘Program’ class:

    class Program
    {
        static readonly Guid IID = Marshal.GenerateGuidForType(typeof(IMyService));

        static void Main(string[] args)
        {
            switch (args[0].ToLower())
            {
                case "listen":
                    {
                        using (RpcServer.CreateRpc(IID, new MyService.ServerStub(new Implementation()))
                            .AddAuthNegotiate()
                            .AddProtocol("ncacn_ip_tcp", "8080")
                            .AddProtocol("ncacn_np", @"\pipe\MyService")
                            .AddProtocol("ncalrpc", "MyService")
                            .StartListening())
                        {
                            Console.WriteLine("Waiting for connections...");
                            Console.ReadLine();
                        }
                        break;
                    }
            }
        }
    }

Step 4 – Sending a Message

Now that we have a working server we need to write a client. The reason for the Main() method above to switch on arg[0] for ‘listen’ is that we are going to use the same program for a client. The client case statement below adds support for an LRPC client call:

                case "send-lrpc":
                    {
                        using (MyService client = new MyService(
                            RpcClient.ConnectRpc(IID, "ncalrpc", null, "MyService")
                            .Authenticate(RpcAuthenticationType.Self)))
                        {
                            MyResponse response = client.Send(
                                MyRequest.CreateBuilder().SetMessage("Hello via LRPC!").Build());
                        }
                        break;
                    }

Once we have added this switch case to the Main routine we wrote we now run one process with the ‘listen’ argument, and another one with the ‘send-lrpc’ argument. We should see the following output in the server process:

Waiting for connections...
DOMAIN\user says: Hello via LRPC!

You may now create two additional case labels, one for “send-tcp”, and one for “send-np”. The only difference between them will be the parameters to the RpcClient.ConnectRpc() api. For TCP/IP we will use RpcClient.ConnectRpc(IID, “ncacn_ip_tcp”, @”localhost”, “8080″), and for named-pipes we would use RpcClient.ConnectRpc(IID, “ncacn_np”, @”\\localhost”, @”\pipe\MyService”). Go ahead and fill those in or not at your choosing.

Step 5 – Authentication

By default the RPC server will allow any user (even anonymous users) to connect. This may work for your needs, this may not. Usually you will want to impersonate the caller and then verify they have access to some resource or are a member of a specific group, etc. To do this in a generic way so that we do not have to place the impersonation code in each method we are going to implement the Google.ProtocolBuffers.IRpcServerStub interface. So let’s create a new class now called Impersonation and it’s going to look a lot like the following:

    class Impersonation : IRpcServerStub
    {
        private readonly IRpcServerStub _stub;

        public Impersonation(IRpcServerStub stub)
        {
            _stub = stub;
        }

        public IMessageLite CallMethod(string methodName, ICodedInputStream input, ExtensionRegistry registry)
        {
            using(RpcCallContext.Current.Impersonate())
            {
                return _stub.CallMethod(methodName, input, registry);
            }
        }

        public void Dispose()
        {
            _stub.Dispose();
        }
    }

Once that has been added we will update our server’s listen routine as follows:

                case "listen":
                    {
                        using (RpcServer.CreateRpc(IID, new Impersonation(new MyService.ServerStub(new Implementation())))
                            .AddAuthNegotiate()
                            ... etc ...

Now every call into every method of MyService implementation on the server will be impersonating the client user. The Rpc layer will also ensure that they are NOT an anonymous user.

Zipping it all up…

Server Options: There are numerous extensibility points on the server and client. There are a few worth covering here. The following is a brief outline of the most important configuration options.

  • RpcServer.EnableMultiPart() – Allows unlimited message lengths to be received over tcp/np connections. By default Windows limits these to around 1mb. To circumvent this limitation the server and client can be configured to send messages in multiple parts. Both client and server must enable this for this to work, and doing so will enable server-side connection state.
  • RpcServer.ExceptionDetails – An enumeration value that determines how much exception detail to return to the client. The default, FullDetails, returns all information in the exception including the stack trace.
  • RpcServer.ExtensionRegistry – Allows registration of proto-buffer ‘extensions’ on both your service and on the transport messages defined in csharp_rpc_messages.proto. This can be used as a side-channel to flow information from the client to server and back again.
  • RpcCallContext.Current
  • – This class provides context information about the caller, protocol, authentication, etc.

  • RpcSession.EnableSessions()
  • – Enables session state, accessed via RpcCallContext.Session for the current call.

Client Options: The following controls the client options:

  • RpcClient.EnableMultiPart() – Allows unlimited message lengths to be sent over tcp/np connections. By default Windows limits these to around 1mb. To circumvent this limitation the server and client can be configured to send messages in multiple parts. Both client and server must enable this for this to work, and doing so will enable server-side connection state.
  • RpcClient.ExceptionTypeResolution – This enumeration controls the exception type resolution when an exception is returned from a server. The default, OnlyUseLoadedAssemblies, will only resolve types that are defined in assemblies that have already been loaded into the client process.
  • RpcClient.ExtensionRegistry – Just as for the server, this allows proto-buffer ‘extensions’ to be registered and used when receiving response messages.
  • RpcClient.CallContext – Provides access to the call context instance associated with this connection. Used with the extension registry this allows you to customize side-channel communications between the client and server.

Connection Caching
It should be noted that it is acceptable and generally useful to cache the RpcClient connection; however, you should be aware that a connection can be closed. RpcClient connections will not retry a failed call and will not attempt to reconnect to a server once disconnected. Due to this it is advisable that if you are caching client connections you should create an implementation of Google.ProtocolBuffers.IRpcDispatch. Delegate the actual RpcClient.ConnectRpc() call and configuration to this object so that it can reconnect at will. Finally use this object as as the parameter to the MyService() constructor instead of directly using the RpcClient.ConnectRpc() result.

zip Download the Sample RPC Project
The project zip file is completely stand-alone. Just extract the contents and open the solution to build.
 

So now that I’ve finally got around to releasing a protocol built on top of the RpcLibrary I thought it would be fun to re-run some benchmarks. The protocol has been around unreleased for almost a year waiting on some required changes in protobuf-csharp-port which have finally been published in a release. The protocol I’m speaking of, Google.ProtocolBuffers.Rpc, uses protobuffer services and the RpcLibrary to provide a full-featured rpc client/server implementation.

Benchmark Process
For the comparison benchmarks I wanted to demonstrate both the power of the RpcLibrary as well as the efficiency of this protobuffer library. So I built this rpc test-harness that spawns a server process and multiple client processes (5 of them) each using multiple threads (3 each). These 15 threads wait on a global signal until everyone is ready and then run 3 times in succession for a fixed duration of 5 seconds. The total numbers of calls made is then divided by the thread’s running duration to produce a calls-per-second. Since this happens 3 times on 15 threads, there are 45 results per test that are then used to produce a worst, average, and best time.

Benchmark Test Data
Each call passes a single object in and returns a single object as a result. The object is a collection of a ‘sample data’ class that has the following data: 32 bytes of random data, those 32 bytes as a base-64 encoded string, a sequentially incremented integer, a double, and a date-time value. The graph title below indicates the number of these records passed in and out for each call. Zero is used for an empty collection/message to test the transport and dispatch speed while only serializing an empty collection. The test was completed for all transport/protocols for message sizes of 0, 10, and 100 records. At 1000 records the test duration was increased to 50 seconds and only ProtoBuf_LRPC and Wcf_TCP were executed.

Transports & Protocols
Most of the transports and protocols used should be obvious enough by the abbreviation name in the charts below; however, there are a few worth calling attention to. All the Wcf_xxx are in-process hosted WCF listeners including the Wcf_Http test. All the ProtoBuf_xxx tests used the RpcLibrary except the ProtoBuf_Wcf test. The ProtoBuf_Wcf used protobuffer protocol serialization on a WCF/TCP transport. The WCF service was basically just a Stream argument, Stream return prototype. I see a lot of people packing protobuffers over WCF without realizing that serialization is not WCF’s only problem. All the tests ending with “_Auth” are fully secured and authenticated connections.

Results with Empty Messages
The top of the blue bar is the worst run’s average calls-per-second, the top of the red bar is the average of all 45 runs, and the top of the green bar is the best calls-per-second of all 45 runs.

Clearly there is almost no comparison here, WCF is running at 1/20th of the speed of the RpcLibrary! That remember is with a completely empty message. I was actually pleased to see that WCF even completed these tests, the last time I tried in .NET 2.0 (WCF 3.0) it dropped client connections and caused them to timeout. This time around running WCF with the full 3.5 sp1 framework it ran flawlessly even if slowly.

It Only Gets Worse for WTF, err WCF
The next two benchmarks only widen the gap between the RpcLibrary and WCF. With just 10 ‘records’ RpcLibrary + Protobuffers outpaces WCF by 25 times, and at just 100 records that jumps to more like 40 times faster! After that it seems to start to level off, 1000 records averaged 40.8 calls-per-second on protobuffers+Rpc, and WCF averaged only 0.91 calls-per-second. That is still a 45 fold increase in performance and, in IMHO, it’s far more acceptable to execute a round-trip in 24 milliseconds instead of 1.1 seconds.

Summary
So the next time you’re looking to stand up an internal service to talk to on the local machine or within the local intranet you should take a close look at alternatives to WCF. It’s wicked fast and I’ve never seen RPC simply ‘hang’ with a broken connection.

 

A few weeks ago I published NuGet packages for version 1.11.924.348.

 
I also managed to get protobuf-csharp-port on board. I absolutely live by protobuffers.

 
Finally I recently published a complete protobuf services rpc layer build on the RpcLibrary and protobuf-csharp-port. Of course much of the code there could be used with any raw transport like WCF, TCP, Named Pipes, etc so check it out. This library is also available on NuGet:

 
If you haven’t tried NuGet you should look at it. If you manage an open source library you really need to get published there. It has allowed me to publish projects that have dependencies without the need to check those dependencies into source control. This reduces the burden on source control, allows for smaller archives, and makes it easier to upgrade dependencies. So trash the binaries from your source control and get started with NuGet.

 

Just published the latest release v1.10.1124.358 of the C# Library hosted on google code. Major new functionality in the form of the new assembly CSharpTest.Net.RpcLibrary. This library wraps PInvoke calls to the Win32 RPC APIs. This allows pure managed C# applications the ability to move byte arrays over the supported rpc protocols. The following is a basic demonstration:

void Test()
{
    //An id to identify the endpoint interface
    Guid iid = Guid.NewGuid();
    using (RpcServerApi server = new RpcServerApi(iid))
    {
        //Allow up to 5 connections over named pipes
        server.AddProtocol(RpcProtseq.ncacn_np, @"\pipe\testpipename", 5);
        //Authenticate via WinNT
        server.AddAuthentication(RpcAuthentication.RPC_C_AUTHN_WINNT);
        //Start receiving calls
        server.StartListening();
        //When a call comes, do the following:
        server.OnExecute +=
            delegate(IRpcClientInfo client, byte[] arg)
            {
                using (client.Impersonate())
                    return new byte[0];
            };

        //For the client, we specify the protocol, endpoint, and interface id to connect
        using (RpcClientApi client = new RpcClientApi(iid, RpcProtseq.ncacn_np, null, @"\pipe\testpipename"))
        {
            //Authenticate as ourselves
            client.AuthenticateAs(RpcClientApi.Self);
            //Call the server
            byte[] response = client.Execute(new byte[0]);
        }
    }
}

Protocols tested include named-pipes, LRPC, and TCP/IP. I’m sure you could get others working as well, but this was enough for a good start.

So someone might start asking “But why would you do this, why not use WCF, TCP Listener, Http Listener, etc?” I’ll try to address these, but you really need to have former experience with RPC to truely understand the benefits…

  • WCF and .Net Remoting are dog slow by comparison, up to 20 times slower.
  • WCF (IMHO) is poorly designed, have you seen anyone provide their own serialization or transport? Nope you havn’t. The original .Net Remoting layer did a much better job in the design; however, it did not provide an out-of-the-box transport layer that was even remotely acceptable.
  • Try it for yourself: A simple test of a WCF server and 10 client processes with 10 threads will fail horribly once the message complexity grows to a reasonable amount (around 100 FileInfo structures should suffice to blow it up).
  • WCF has it’s place, it’s easy, has a wider platform support, but it’s just not very good in reliability and it’s throughput and parallel processing capabilities are far from ideal.
  • Custom listeners (Tcp/NetPipe/Http) can also be useful; however, they simply cannot provide the robust channel that RPC will provide over the same protocols.

 
The next objection that seems obvious enough: your left with only a transport. There is no ability to serialize ‘messages’ from client to server and back. This is by choice, the transport provided here is intended to be only that, a transport. It moves bytes from one process to another, that is all it does. I’m currently working on final touches to a proto-buffer wrapper around this transport using Jon Skeet’s dotnet-protobufs. It’s fast, reliable, secure, and extensible… all the things I could only wish Microsoft had delivered in .Net 1.0.

Anyway, enough ranting on the failings of Microsoft and .Net RPC. If your looking for a descent transport, LRPC support, or OOP C++ integration, this might be what your after.