Library

 

Essentially a collection of some basic utilities I’ve been using. Most of these were built as I ever-so-slowly make my way towards a final and (as of yet) unnamed goal. This library has been used on several projects from the JiraSvn integration to the SslTunnel service.

Here is rundown of some key capabilities provided:

  • AppConfig – An implementation of UserSettings as a ConfigurationManager extension.
  • Bases – Base classes for IDisposable, IComparable, and IEquatable.
  • Cloning – A generic deep clone implementation for C# objects that may or may not be serializable.
  • Collections – Some collections classes what else?
  • Commands – Provides a CommandInterpreter for processing of command-line args.
  • Crypto – Some utilities around encrypting/decrypting data for the local user or machine.
  • Delegates – Provides a delegate wrapper to ensure thread-safety in WinForms.
  • Formatting – Various encoding formats for binary data.
  • Html – A regular expression xml/html parser and DOM object model.
  • Interfaces – A small collection of commonly needed interfaces.
  • IO – A collection of Stream utilities and derivations.
  • IpcChannel – An event-based message broadcast class for inter-process communication.
  • Processes – Provides a ProcessRunner utility class to capture output from a process.
  • Reflection – Reflection utilities for reading/writings fields & properties.
  • Serialization – Provides string-based storage serialization of name/value pairs.
  • Synchronization – A collection of Reader/Writer lock implementations sharing common interfaces.
  • Threading – Threaded work queues for delegates and similar classes.
  • Utils – general utility methods for file and string manipulation.
Links
 

A few weeks ago I published NuGet packages for version 1.11.924.348.

 
I also managed to get protobuf-csharp-port on board. I absolutely live by protobuffers.

 
Finally I recently published a complete protobuf services rpc layer build on the RpcLibrary and protobuf-csharp-port. Of course much of the code there could be used with any raw transport like WCF, TCP, Named Pipes, etc so check it out. This library is also available on NuGet:

 
If you haven’t tried NuGet you should look at it. If you manage an open source library you really need to get published there. It has allowed me to publish projects that have dependencies without the need to check those dependencies into source control. This reduces the burden on source control, allows for smaller archives, and makes it easier to upgrade dependencies. So trash the binaries from your source control and get started with NuGet.

 

A very minor update to the library this past weekend.

  • Addition of Cyrpto.SecureTransfer to provide file transfers via shared public keys.
  • The Crypto.AESCryptoKey now has ToArray() and FromBytes() like other keys.
  • HashStream can now aggregate read/write calls to actual storage stream while computing the hash.
  • The Crypto.Hash class received a new method, Combine(…)
  • Html.XmlLightElement and related classes are now fully modifiable.
  • BPlusTree.Options now supports a ReadOnly property to ensure no writes at the file handle level.
 

Last year I ran across this 2009 post by Bruce Schneier entitled “Another New AES Attack“. It got me thinking about and dissecting the Rijndael algorithm which most of you know as AES (Advanced Encryption Standard). This research surprised me, I found that AES has only three variants. These variants are best known by their key sizes of 128, 192, or 256 bits, but they also specify the rounds, or iterations, made during the transformation. The surprise was the underlying algorithm, Rijndael, has no such limitation. Funny enough this is exactly Schneier’s recommendation from the above article:

… At this point, I suggest AES-128 at 16 rounds, AES-192 at 20 rounds, and AES-256 at 28 rounds. Or maybe even more; we don’t want to be revising the standard again and again.

Then today, two years later, another related post from the same author “New Attack on AES” with the same recommendation.

So what does this mean for me and my software?

Well that depends on how sensitive the data is, how long you intend to store it, and how accessible is the stored cipher.

For instance, SSL uses AES-128 after the initial key exchange; however, this does not mean SSL is broken (ok it is, but for other reasons). Since the SSL session key is short-lived, and the data is not stored anywhere (well, not suppose to be), there really isn’t a major concern here. The same is likely true even for your own software, transient data encrypted for short periods of time.

Well wait a minute you ask, “what about those credit cards I have encrypted in my database?” Frankly this still isn’t much of a concern. Credit cards actually aren’t that ‘sensitive’ to begin with, they get compromised all the time (Thanks Sony). Further most credit cards expire within two years, which is about how long the most gratuitously over optimistic attack could occur. So no, I would not reinvent the way you store credit cards. I would however make sure your not storing the encryption key in that database :)

So why would I concern myself with this? Well we are well within the era of SaSS/Cloud computing and often enough we need to encrypt and store data for our users. So we have to ask ourselves how sensitive this data is, how much damage can be done by accessing the data. Thinking of services like LastPass and 1Password, they store potentially sensitive data for long periods of time. Of course if I were going to attack those services, I’d likely go after the pass phrase, not the raw crypto key, but still. These services could easily benefit from extended rounds of Rijndael algorithm.

Another classic place to improve is locally encrypted storage. This was what I was playing with when I wrote the ModifiedRijndael class. It allows you to specify key sizes from 128~4k in 64bit increments, and any compatible number of Rounds. So if you want to play around with unlocking the Rijndael algorithm from the AES constraints try it out:

        ModifiedRijndael r = new ModifiedRijndael();
        r.BlockSize = 128;
        r.KeySize = 128;
        r.Rounds = 20;
        r.GenerateIV();
        r.GenerateKey();
        ICryptoTransform alg = r.CreateEncryptor();

Sum it up: The above articles and information is really only just that right now, information. I don’t see many people needing this kind of added security when encrypting data. A classic place for me to want something like this would be when uploading a file to something like DropBox. The problem is, I would likely generate the key from a password. The password is much easier to attack than the AES-128 algorithm. So until we can solve the password entropy issue there just doesn’t seem like much point in moving beyond AES for *most* cases.

 

The help site doesn’t suck as bad as did before. Make me feel better and take a look:

http://help.csharptest.net/

I’ll know if you don’t.

So I finally gave up on my last effort at getting help automated.  To recap my struggle: basically Sandcastle, the original help system I used, was a complete flop.  I spent days trying to get the thing to work and had untold nightmares working with it.  I think I only ever pushed one copy with it.  Then I thought I’d play around with Reflector.NET and build my own help generator that included the actual source code.  It actually worked pretty well although it was somewhat less aesthetically pleasing to view.  Trouble is Reflector.NET is no longer free and I’m a little upset by their choice.

Finally I needed to produce some documentation for work and needed a ‘real’ solution.  The API we were documenting at work had a rather unique need, the dll was generated from a COM import and had no source files.  So I went searching for a tool that could give me a WYSIWYG help editor for .NET assemblies.

I FOUND THE KILLER HELP APP!

Stumbling across Document! X was an absolute stroke of good fortune.  It did everything I needed for the project at work and I immediately knew I needed a copy for this project!  A little on the pricey side I requested an open-source license and they granted my request.

The results of Document! X I think speak for themselves so now go back and click on the help link before reading more.  I’ve been working with this product for almost a month now and I have to say I’m impressed by several aspects of it.  One of the grandest things is that all their templates are well laid out in a single ‘configurations’ folder.  A few personal preferences drove me to make a few tweaks here and there and it was easy and straight-forward to do.  Of course not every product is perfect and Document! X is no exception, but it is the best I’ve worked with since NDoc and FAR more capable.  My only real gripe is that the embedded source editor is not a text-area and I keep winding up with html getting pasted in there.  Still it’s a pleasure to work with, I didn’t have to learn anything new, just picked it up and started working.  Honestly this is a solid 9 out of 10 on my scale, so if you need documentation try it out.

 

Just published the latest release v1.10.1124.358 of the C# Library hosted on google code. Major new functionality in the form of the new assembly CSharpTest.Net.RpcLibrary. This library wraps PInvoke calls to the Win32 RPC APIs. This allows pure managed C# applications the ability to move byte arrays over the supported rpc protocols. The following is a basic demonstration:

void Test()
{
    //An id to identify the endpoint interface
    Guid iid = Guid.NewGuid();
    using (RpcServerApi server = new RpcServerApi(iid))
    {
        //Allow up to 5 connections over named pipes
        server.AddProtocol(RpcProtseq.ncacn_np, @"\pipe\testpipename", 5);
        //Authenticate via WinNT
        server.AddAuthentication(RpcAuthentication.RPC_C_AUTHN_WINNT);
        //Start receiving calls
        server.StartListening();
        //When a call comes, do the following:
        server.OnExecute +=
            delegate(IRpcClientInfo client, byte[] arg)
            {
                using (client.Impersonate())
                    return new byte[0];
            };

        //For the client, we specify the protocol, endpoint, and interface id to connect
        using (RpcClientApi client = new RpcClientApi(iid, RpcProtseq.ncacn_np, null, @"\pipe\testpipename"))
        {
            //Authenticate as ourselves
            client.AuthenticateAs(RpcClientApi.Self);
            //Call the server
            byte[] response = client.Execute(new byte[0]);
        }
    }
}

Protocols tested include named-pipes, LRPC, and TCP/IP. I’m sure you could get others working as well, but this was enough for a good start.

So someone might start asking “But why would you do this, why not use WCF, TCP Listener, Http Listener, etc?” I’ll try to address these, but you really need to have former experience with RPC to truely understand the benefits…

  • WCF and .Net Remoting are dog slow by comparison, up to 20 times slower.
  • WCF (IMHO) is poorly designed, have you seen anyone provide their own serialization or transport? Nope you havn’t. The original .Net Remoting layer did a much better job in the design; however, it did not provide an out-of-the-box transport layer that was even remotely acceptable.
  • Try it for yourself: A simple test of a WCF server and 10 client processes with 10 threads will fail horribly once the message complexity grows to a reasonable amount (around 100 FileInfo structures should suffice to blow it up).
  • WCF has it’s place, it’s easy, has a wider platform support, but it’s just not very good in reliability and it’s throughput and parallel processing capabilities are far from ideal.
  • Custom listeners (Tcp/NetPipe/Http) can also be useful; however, they simply cannot provide the robust channel that RPC will provide over the same protocols.

 
The next objection that seems obvious enough: your left with only a transport. There is no ability to serialize ‘messages’ from client to server and back. This is by choice, the transport provided here is intended to be only that, a transport. It moves bytes from one process to another, that is all it does. I’m currently working on final touches to a proto-buffer wrapper around this transport using Jon Skeet’s dotnet-protobufs. It’s fast, reliable, secure, and extensible… all the things I could only wish Microsoft had delivered in .Net 1.0.

Anyway, enough ranting on the failings of Microsoft and .Net RPC. If your looking for a descent transport, LRPC support, or OOP C++ integration, this might be what your after.

 

So first off this is not new, it’s been around for a long, long while. One issue I have with IDisposable is that for whatever reason .Net did not include a base class for handling the details… I admit you may not always be able to leverage it since you may have a base class, [...]

 

This one is actually something of a triviality, yet it can be very useful at times. Now a word of caution is due here, this object is built on System.Threading.Timer and should not be used to enqueue 12 million work items. Yet if you need to perform a simple task at some time in the [...]

 

Most people I’ve seen online compute a simple hash of password + salt for persistence and authentication. This is the accepted standard in a straight-forward solution: byte[] Hash(string password) { byte[] pass = System.Text.Encoding.UTF8.GetBytes(password); //Create the salt to use byte[] salt = new byte[32]; new RNGCryptoServiceProvider().GetBytes(salt); //Create the hash of password and salt HashAlgorithm hashAlgo [...]

 

So recently I’ve been working heavily with some of the cryptography stuff in .Net and adding a large amount of it to my open source library. One of the many things I needed to perform was simply encrypting and decrypting a piece of data with a password. It seems everyone out there is using Rfc2898DeriveBytes [...]

 

Changes in this version: Library.Crypto namespace was added with a fairly complete cryptography API (at least what I needed) including: Added Library.Crypto.WhirlpoolManaged a managed implementation of the whirlpool hash function. Added Library.Crypto.Password to wrap up the complexities of using a password for authentication and/or encryption. Library.IO namespace was added to include several new stream derivations [...]