This ‘helpful’ (using this term loosely) post is for those you considering adding multi-threaded code to your application.  The intended audience is developers with less than mastery-level (10+ years) multi-threading experience.

The first question your up against is:

Do I need multi-threading in my application?

The answer is an emphatic NO you don’t.  If you accept this answer you can quit reading now; otherwise I have a different question for you:

Would you be willing to voluntarily walk across across 100 yards of molten lava, bare foot, carrying a 250lb backpack, uphill, with a blind fold on, all the while ignoring the perfectly good bridge over the lava you simply ignored because it was too easy?

Seriously though, avoid at any cost as much as you can adding multi-threading code.

Common ramifications of threading

  • Most applications that employ threading find a great majority of their bugs in the threading code.
  • Almost always the difficult/impossible to reproduce issues stem from broken threading.
  • More often that not, the application actually performs worse once you start multi-threading.
  • Most of the time threading wasn’t truly *required* in the first place.

If you still think multi-threading isn’t that hard read this article on a simple thing like initializing a static variable in a thread-safe manner:  http://www.yoda.arachsys.com/csharp/singleton.html

So you still want to do multi-threading?

Well, I tried and failed.  Now you need to go learn EXACTLY what the following are used for and when/why you would use them (and generally in this order):

  1. Thread
  2. ThreadPool
  3. ManualResetEvent
  4. AutoResetEvent
  5. EventWaitHandle
  6. WaitHandle
  7. Monitor
  8. Mutex
  9. Semaphore
  10. Interlocked
  11. BackgroundWorker
  12. AsyncOperation
  13. lock Statement
  14. volatile
  15. ThreadStaticAttribute
  16. Thread.MemoryBarrier
  17. Thread.VolatileRead
  18. Thread.VolatileWrite

 

Once you fully understand these your ready write what will prove to be a lot of broken multi-threaded code. Go wild, have fun, and enjoy it… just don’t put it in a product.  Failure is the ultimate, and often only, teacher.  Such is the case with multi-threaded code.  If you want to expedite your failures, use the CHESS program by Microsoft.Some things to consider:

While writing anything thread-safe, ask yourself the following questions:

  1. What can happen if two threads are executing each statement in step with each-other?
  2. Do I really need this lock()?
    1. You need a lock if you are going read AND write to memory that other threads will also access.
    2. Consider using Interlocked instead.
  3. What’s the least amount of code that MUST be in the lock()?
  4. Can I make this object/state immutable and avoid the locking?
  5. What’s the harm if I don’t lock it?
  6. Is there any way for a caller to have code execute inside the lock?
    1. Do not call delegates inside a lock unless you created it.
    2. Do not call methods/properties inside a lock on a class you didn’t create or don’t control.
    3. Never invoke to a Control from within a lock.
  7. Review all members accessed and changed inside a lock for one of these:
    1. Was the variable declared ‘volatile’?
    2. Did I use Thread.VolatileRead/Write?
    3. Did I use Thread.MemoryBarrier?
  8. Are you using System.Threading.Thread.MemoryBarrier correctly?
  9. If multiple locks are involved are they always obtained in the same order?
  10. Don’t use a private member (or expose if you must) to synchronize as this prevents ordering locks.

 
Things you may not need to make thread-safe:

1. Quite commonly you find people locking on initialization of a property when performing a lazy-load:

		private volatile object _data;
		public object Data
		{
			get
			{
				if (_data != null) return _data;
				lock (this)
				{
					if (_data == null)
						_data = new object();
				}
				return _data;
			}
		}

This is usually not necessary since the worst that happens is the data is loaded twice.  It depends greatly on what that object is and can do, but usually it’s safe to load two instances.  Think about it without concern for performance (the likely-hood is too small to be a performance impact). If you can load/create the data twice without causing issue then just do this instead:

		public object Data
		{
			get { return _data ?? _data = new object(); }
		}

2. Consider pre-fetching instead of lazy loading with shared state.  This can then be performed in the constructor which is only on a single thread.

3. Immutable objects (objects that do not change state) do not normally need locks to access.

4. Fire-and-forget objects work best, start a task and don’t wait for it.  You defeat the purpose of multi-threading with synchronization.

And finally, yes you should test it but don’t depend upon it. Get the whole team together if needed to do a line-by-line walk of the code. I’m sure I’ve missed a lot of things here, but this should be a good start.

 

I’ve seen many a question on stackoverflow and other places about running a process and capturing it’s output. Using the System.Diagnostics.Process correctly is not easy and most often it’s done wrong.

Some common mistakes with System.Diagnostics.Process:

  1. Not capturing both output streams (error & output)
  2. Not redirecting input can cause applications to hang
  3. Not closing redirected input can cause applications to hang
  4. Not calling BeginOutputReadLine/BeginErrorReadLine when using events
  5. Using OutputDataReceived/ErrorDataReceived without waiting for null
  6. Not checking for null in OutputDataReceived/ErrorDataReceived handlers
  7. Forgetting to set EnableRaisingEvents = true; when using the Exited event
  8. Forgetting ErrorDialog, CreateNoWindow, or UseShellExecute settings
  9. Incorrect handling of StandardOutput or StandardError stream readers

 

So with this said, here are some basic guidelines:

  1. Use the OutputDataReceived/ErrorDataRecieved events NOT the StandardOutput or StandardError. This will save you a lot of headache and needless thread management.
  2. Always capture all output AND input, if you don’t plan to provide input, close the stream immediately.
  3. Your process isn’t done until it exited AND you have read all the data. OutputDataReceived CAN AND WILL be fired after a call to WaitForExit() returns. You will need wait handles for each output stream and set the wait handle once your receive (null) data.

 

Additional Resources:

 

Let’s face it, using the System.Diagnostics.Process to run a process and capture the output is not as easy as it should be. People often get it wrong without realizing it. Adding to the complexities can be reading both the standard output as well as the standard error stream. To put this to bed once and for all here is the ‘right’ way all wrapped up in a helper class ProcessRunner.

Usage Example:

	using CSharpTest.Net.Processes;
	partial class Program
	{
		static int Main(string[] args)
		{

			string path = "C:\\MyProject";
			ProcessRunner run = new ProcessRunner("svn.exe", "update", "{0}");
			run.OutputReceived += new ProcessOutputEventHandler(run_OutputReceived);
			return run.RunFormatArgs(path);
		}

		static void run_OutputReceived(object sender, ProcessOutputEventArgs args)
		{
			Console.WriteLine("{0}: {1}", args.Error ? "Error" : "Output", args.Data);
		}
	}

 

 

For many years (more than I care to remember) I have continually written code to provide command-line behavior over and over again. On occasion I would get fancy in one project or another and build something sorta-generic; however, every time it came to creating the next command-line tool I started from scratch. Finally I said “NO MORE!”

The result of that was the CommandInterpreter class. What does it do? Well you give it a System.Type and it will create ‘commands’ from all public static methods, or give it an object instance and it will create ‘commands’ from all the public instance methods. With these commands you can have a fully operational command-line application complete with help in one line of code:

	using CSharpTest.Net.Commands;
	static class Program
	{
		static void Main(string[] args)
		{
			new CommandInterpreter(typeof(Commands)).Run(args);
		}
	}

For this example let’s pretend we have the following ‘Commands’ class:

	static class Commands
	{
		public static void DoSomething(string whoAreYou, int repeatMessage)
		{
			for (int i = 0; i < repeatMessage; i++)
				Console.WriteLine("Hello {0}, how are you today?", whoAreYou);
		}
	}

Now let’s see what we can do:

C:\Test>Sample.exe help
Commands:
  DOSOMETHING:  Some descriptive text
      HELP:  Gets the help for a specific command or lists available commands.

C:\Test>Sample.exe help dosomething

DOSOMETHING [/whoAreYou=]String [/repeatMessage=]Int32

descriptive text

Arguments:

  [/whoAreYou=]String argument description
  [/repeatMessage=]String argument description

C:\Test>Sample.exe dosomething roger 2
Hello roger, how are you today?
Hello roger, how are you today?

 
So this is where your jorney begins and mine ends, there is so much more you can do with this class I can’t put it all here … even though I will still try:

  • Run interactively with the console via CommandInterpreter.Run(Console.In)
  • Use global environment settings by using public properties on the object/type.
  • Inject properties into arguments via nmake-like $(OptionName) macro expansion.
  • Implement a command as a interface (ICommand, IOption) and do anything.
  • Imlement custom argument parsing by declaring yourMethod([AllArguments]string[] args)
  • Environment.ExitCode is set for you on exceptions, no need to catch{} anything.
  • Throw an ApplicationException to output to Console.Error only the exception’s message.
  • Run complete scripts via Run(TextReader in).
  • Aggregate commands by accepting an ICommandInterpreter and calling Run(…).
  • Inject an ICommandFilter implementation to pre/post process every command.
  • Provide arguments by ordinal occurrence or explicitly by /name=value or -name:value
  • Accept multiple values for an argument by using (string[] name)

 

Attributes used:
  • [System.ComponentModel.DisplayName("x")] – renames a command.
  • [System.ComponentModel.Description("x")] – defines help for methods and arguments.
  • [System.ComponentModel.Browsable(false)] – hides a methodfrom the help command.
  • [System.ComponentModel.DefaultValue(0)] – default value for a property or argument.
  • [AliasName("x")] – alternative name(s) for commands, options, and arguments.
  • [AllArguments] string[] – argument receives all original arguments.
  • Declare a method as an ICommandFilter with the [CommandFilter] attribute.

 

I know your interested, so go get the source code or download the binaries.
 

A brief how-to on providing user settings in your application.

First, if you want to provide defaults, you want to add the following to the application’s configuration:

  

Notice there are two levels of the <add> tag, one is ‘global’ settings, the other is segregated off in a named section.

Reading this information is along the following lines:

			//reads the current user's settings
			UserSettingsSection settings = UserSettingsSection.UserSettings;
			//access a global setting:
			Assert.IsTrue(settings["a"] == "b");
			//access a setting from within the section "c"
			if(settings.Sections["c"] != null)
				Assert.IsTrue(settings.Sections["c"]["x"] == "y");

To update the information you need to perform the following:

			//Open the configuration
			Configuration cfg = ConfigurationManager.OpenExeConfiguration(ConfigurationUserLevel.PerUserRoamingAndLocal);
			UserSettingsSection settings = UserSettingsSection.UserSettingsFrom(cfg);
			//Push the settings in
			settings["a"] = "b";
			settings.Sections["c"] = new UserSettingsSubSection();
			settings.Sections["c"]["x"] = "y";
			//save the configuration
			cfg.Save();

Get the source code or download binaries.

 

After writing my last article I began to wonder what people ‘in-the-know’ thought the advantages of TDD vs integration testing were. So I quickly turned to my new favorite site stackoverflow. After reviewing several questions I came across this one entitled “Is Unit Testing worth the effort?“, and the accepted answer had 113 votes. So [...]

 

I was viewing the comments on a recent post entitled Integration Tests Are a Scam when I ran across this: Integration tests are needed A Mars rover mission failed because of a lack of integration tests. The parachute subsystem was successfully tested. The subsystem that detaches the parachute after the landing was also successfully (but [...]

 

Changes in this version: Added a command-line interpreter and parser under Library.Commands Added a WinForms cross-threaded event delegate class that prevents deadlocking Added Library.Processes.ProcessRunner utility class for spawning a process and correctly collecting the Output Added a few FileUtils to allow searching the environment path and granting full access on a file for a well-known [...]