byte[] Hash(string password)
    {
        byte[] pass = System.Text.Encoding.UTF8.GetBytes(password);
        //Create the salt to use
        byte[] salt = new byte[32];
        new RNGCryptoServiceProvider().GetBytes(salt);
        //Create the hash of password and salt
        HashAlgorithm hashAlgo = new SHA256Managed();
        hashAlgo.TransformBlock(salt, 0, salt.Length, salt, 0);
        hashAlgo.TransformFinalBlock(pass, 0, pass.Length);
        byte[] hash = _hashAlgo.Hash;
        hashAlgo.Initialize();
        //Copy the combined salt + hash to a single array
        byte[] result = new byte[salt.Length + hash.Length];
        Array.Copy(salt, result, salt.Length);
        Array.Copy(hash, 0, result, salt.Length, hash.Length);
        return result;
    }

The only deviation i’ve seen from this is to either use existing data (say a primary key in the database) for the salt, or to ‘hide’ the salt at some offset in the result. Both of these ideas are valid yet amount to not much more than a little obfuscation. I don’t recommend it.

Now if you REALLY want to secure your passwords and/or protect against brute force attacks there is another approach. The approach is quite common in generating crypto keys from passwords but can just as easily be used for hashing the password. The idea/concept is expressed in RFC2898 by the introduction of the iteration count. By introducing this into the password hash we increase the computational complexity required to test a password by several orders of magnitude. The straight-forward way of achieving this in the BCL is as follows:

    byte[] Hash(string password)
    {
        //Create the salt to use
        byte[] salt = new byte[20];
        new RNGCryptoServiceProvider().GetBytes(salt);
        //Combine the salt with the iteration-count
        byte[] iterationBytes = BitConverter.GetBytes(1010);
        byte[] iterationSalt = (byte[])salt.Clone();
        for (int i = 0; i < iterationSalt.Length; i++)
            iterationSalt[i] ^= iterationBytes[i % iterationBytes.Length];
        //Create the hash of password and salt
        DeriveBytes deriveBytes = new Rfc2898DeriveBytes(password, iterationSalt, 1010);
        byte[] hash = deriveBytes.GetBytes(20);
        //Copy the combined salt + hash to a single array
        byte[] result = new byte[salt.Length + hash.Length];
        Array.Copy(salt, result, salt.Length);
        Array.Copy(hash, 0, result, salt.Length, hash.Length);
        return result;
    }

This will work very well but will be more costly to calculate than the first algorithm. I would not want to use this in a system that verifies passwords on every request (i.e. Basic Auth over SSL); however, for token-auth systems this will prove much more durable to brute force attacks. By combining the iteration count to the salt (or key) you make the attacker repeat the entire iteration sequence for each iteration count attempted. This makes it significantly more difficult for them to steal the password hashs and crack the passwords without also stealing the software or otherwise knowing the iteration count. Even knowing the iteration count it’s now much more expensive to compute each attempt at matching an entry in their password dictionary.

Recently I’ve added the PasswordHash class to wrap up some of this behavior. Note that this class does not combine the iteration count with the salt, yet still provides significant security benefits over a simple hash. ]]> http://csharptest.net/?feed=rss2&p=417 http://csharptest.net/?p=413 http://csharptest.net/?p=413#comments Tue, 08 Jun 2010 20:32:36 +0000 rogerk http://csharptest.net/?p=413 So recently I’ve been working heavily with some of the cryptography stuff in .Net and adding a large amount of it to my open source library. One of the many things I needed to perform was simply encrypting and decrypting a piece of data with a password.

It seems everyone out there is using Rfc2898DeriveBytes by now and not using the older PasswordDeriveBytes. So it’s use is simple enough, construct it with a password, ask it for a number of bytes, and off you go. When things get interesting is correctly providing salt to the derived bytes algorithm. I didn’t want to deal with this all over my code and thus built the PasswordKey class. It auto-magically prepends the salt to the stream when encrypting data. When decrypting data it reads the salt, generates the key, and then decrypts the data.

At first everything went well, it seemed to work and, as per my norm, I quickly got my unit tests pounding on the implementation. Somewhere along the way though I must have used the Rfc2898DeriveBytes class in a way that was not expected as when I began running in release (optimized) mode I started seeing strange results. The tests were generating 32-bit keys with the same password, iteration, and salt combinations and were generating different keys? The funnest part was they only differed after the first 20 bytes. How could this be? If the input was wrong there is almost no likelihood the first 20 bytes would be the same, so how did I fail? I looked and looked, googled and googled some more. Read nearly every line of code in the class using Reflector.Net, nothing. Still to this day I don’t know why it’s breaking, I do know I can get two identical 20-byte keys, or two identical 40-byte keys, but I still get two different 32-byte keys… What should I do? Well I came up with two possible solutions:

1. The first ‘fix’ for this issue resulted in the PBKDF2 class that derives from Rfc2898DeriveBytes. Then by overriding the GetBytes() method I can always pass a value of 20 to the actual GetBytes() base method:

    public override byte[] GetBytes(int cb)
    {
        byte[] buffer = new byte[cb];
        for (int i = 0; i < cb; i += 20)
        {
            int step = Math.Min(20, cb - i);
            Array.Copy(base.GetBytes(20), 0, buffer, i, step);
        }
        return buffer;
    }

2. Then I started debating the value of using a 20-byte hash to generate 32 bytes of data. Seemed like this algorithm was just as applicable with any size hash right? Sure enough… So the second, and finally preferred solution was to recreate the algorithm with a variable hash algorithm. This gave birth to the HashDerivedBytes<THash> implementation which accepts THash to be any HMAC (Keyed hash) derived implementation. With this I’m able to force the use of the managed SHA256 hash routine and elevate the iterations by nearly an order of magnitude without substantial performance loss. In the end I’m pleased with the implementation and can verify it’s compliance with the original algorithm by comparing it’s output using SHA1 with that of the original Rfc2898 and produce compatible results.

Anyway if you have problems with the Rfc2898DeriveBytes class I’d like to hear from you just to make sure I’m not crazy. And now you know of two possible solutions if you do have problems ;)

• Library.Crypto namespace was added with a fairly complete cryptography API (at least what I needed) including:
  • Added Library.Crypto.WhirlpoolManaged a managed implementation of the whirlpool hash function.
  • Added Library.Crypto.Password to wrap up the complexities of using a password for authentication and/or encryption.
• Library.IO namespace was added to include several new stream derivations including:
  • Added Library.IO.SegmentedMemoryStream for memory streaming while avoiding LOH allocations.
  • Added Library.IO.TempFile to manage temp files and remove them when disposed.
  • Added Library.IO.ReplaceFile to transact replacing a file.
• Library.Html namespace was added to help with manipulation of html and xhtml:
  • Added Library.Html.XhtmlValidation will use the w3c xhtml 1.0 DTDs to validate xhtml files.
  • Added Library.Html.HtmlLightDocument to a provide fast DOM parsing of HTML using regular expressions.
  • Added Library.Html.XmlLightDocument to a provide fast DOM parsing of XML using regular expressions.

http://code.google.com/p/csharptest-net/downloads/detail?name=SvnPluginInstall-x64.msi

Let me know if you have any difficulties.

What is the best comment in source code you have ever encountered?

There are a lot of great quotes in there. If you haven’t seen it already it worth a read when you want to kill some time. I’d also add my own to the list:

    // The DM warns the following encounter is EL25 and should only be attempted
    // by Epic level parties.

Before I continue I will again state that I don’t recommend this approach. It will be many times slower than a properly implemented Clone() and can cause rather strange side-effects. For instance, deep-cloning a tree of objects that hold reference to a singleton service class would result in cloning that singleton. This problem simply stems from the fact that no generic code can have enough knowledge about the internal structure of classes to be able to accurately guess when and when not to clone a member reference. Thus if you going to code the logic of which members to clone or not, it’s best done in the objects themselves and is the primary reason I say a generic deep clone is a bad idea. Even if you don’t currently have a reference to a singleton now, what happens when someone adds one to the object graph? How will your application behave?

So with this established as a bad idea it is with great trepidation that I demonstrate how to achieve this. To start with let’s demonstrate usage in the Library:

using CSharpTest.Net.Cloning;

    //Uses serialization API if available or member-wise if non-serializable
    using (ObjectCloner cloner = new SerializerClone())
        fooCopy = cloner.Clone(foo);
    // 3.5 can use extension method instead of the code above:
    fooCopy = foo.DeepClone();

    //To ignore serialization routines, use the following:
    using (ObjectCloner cloner = new MemberwiseClone())
        fooCopy = cloner.Clone(foo);

Implementation details, there are not a lot to share, you can see the source easily enough at:
http://code.google.com/p/csharptest-net/source/browse/#svn/trunk/src/Library/Cloning

There are a few things worth pointing out. The ObjectCloner provides a base-class that handles the object graph and a few well-known types (delegates, arrays, primitives, etc). Derived from this is the MemberwiseClone class which provides a basic ‘memory copy’ type of copy operation. Again deriving from this is the SerializerClone which implements a pseudo serialization routine by attempting to mimic the .Net serializer’s default implementation. This results in three possible types of copy operations:

1. If ISerializable is implemented it will be used, allowing customization of the copy.
2. If [Serializable] decoration is present the FormatterServices.GetSerializableMembers is used to allow respecting the [NonSerialized] attribute.
3. Lastly if the object doesn’t support serialization a straight member-by-member copy is made.

This continues throughout the object graph until all object members have been copied into the new tree. Finally if appropriate calls to IDeserializationCallback will be made just prior to the top-level instance being returned.

One thing of note is that the object graph is made public to allow preventing some instances from being copied. For instance if you want to copy foo’s complete object graph except for any instances of ‘bar’ then you would do this:

using CSharpTest.Net.Cloning;

    using (ObjectCloner cloner = new SerializerClone())
    {
        // maps instances of bar to itself:
        cloner.Graph.Add(bar, bar);
        fooCopy = cloner.Clone(foo);
    }

• CSBuild initial release - a command-line compilation utility that drives MSBuild to compile designated project files.
• Added Library.Cloning namespace to support deep-object cloning of any object using either memberwize or serializable copy.
• Added Library.Collections.ReadOnlyList to provide a read-only collection interface and implementation.
• Added Library.Collections.OrdinalList to provide a collection of integers stored as a bit-array that can be operated on as a set (intersect/union/etc).
• Added Library.Collections.SetList to provide a generic collection of that can be operated on as a set (intersect/union/etc).
• CommandInterpreter can now read SET operations from stream, also added an IgnoreMember attribute.

http://msdn.microsoft.com/en-us/library/system.runtime.serialization.formatterservices_members.aspx

It provides several helpful methods in this case that are aware of custom serialization options like [NonSerialized]. In the case of an object not being marked [Serializable] you have to create field list manually. Here is an example shallow object clone:

using System.Runtime.Serialization;
using System.Reflection;

    static class Clonable
    {
        public static T Clone<T>(this T instance)
        {
            object copy;
            Type type = instance.GetType();

            if (instance is ICloneable)
                return (T)((ICloneable)instance).Clone();

            List<MemberInfo> fields = new List<MemberInfo>();
            if (type.GetCustomAttributes(typeof(SerializableAttribute), false).Length == 0)
            {
                Type t = type;
                while (t != typeof(Object))
                {
                    fields.AddRange(t.GetFields(BindingFlags.Public | BindingFlags.NonPublic | BindingFlags.Instance | BindingFlags.DeclaredOnly));
                    t = t.BaseType;
                }
            }
            else
                fields.AddRange(FormatterServices.GetSerializableMembers(instance.GetType()));

            copy = FormatterServices.GetUninitializedObject(instance.GetType());
            object[] values = FormatterServices.GetObjectData(instance, fields.ToArray());
            FormatterServices.PopulateObjectMembers(copy, fields.ToArray(), values);

            return (T)copy;
        }
    }

As with anything, use this carefully. This method run against some objects will cause native handles to be copied and can then lead to application crashes in unmanaged code.

If you need a deep clone things get a lot more complicated. I don’t recommend it; rather you should insist that objects are decorated with [Serialiable] and simply use serialization to copy the object. That being said, I will post a working deep-clone as a follow up to this post.

Hi, my name is Roger and I’ve been video game free for 8 years. In that time I’ve fallen off the wagon a few times; however, they were brief episodes. As a recovering gameaholic this video really strikes home for me. The message to gamers is spot-on IMO, I just wish those still addicted could truly ‘hear’ what he is saying. See for yourself:

http://www.youtube.com/watch?v=YKBRG_QgEAM

Aside from the message alone, I also want to complement the author on the quality of the video. He is obviously talented at more than just button mashing.