Agile Software Design, Architecture and Planning Tools

Dr. Dobbs published the Jolt Awards for Design, Architecture and Planning Tools today. The Jolt judges make a very important introductory point before discussing the specific winners.

“On large projects, it can be difficult to state requirements, do the design work, and still maintain Agile's orientation towards accepting — even welcoming — new changes from users.

“Those issues notwithstanding, I've noticed that most large or mid-sized projects with mission-critical implications invariably do indeed gather requirements carefully and design the product accordingly. Agile, if it's in use in the organization, typically is most evident in the coding and testing stages.

“Tools that can capture requirements, help illustrate and validate design, and plan the implementation are still very much needed.“

One of the greatest misunderstandings of Agile software development is the assumption that if one thing is valued over another, the thing with lesser value has no value. Let us review the manifesto:

Manifesto for Agile Software Development

We are uncovering better ways of developing
software by doing it and helping others do it.
Through this work we have come to value:

Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan

That is, while there is value in the items on
the right
, we value the items on the left more.

I have added my own emphasis to highlight the point that is so often overlooked by Agile critics and enthusiasts alike. The critic decries Agile for eliminating design, architecture and planning, and yet Agile does nothing of the sort. The enthusiast rushes to write code thinking that Agile is a magic bridge over the need to endure the tedium of design, architecture and planning, and yet, nothing could be further from the truth.

After all, how can you set out to intelligently implement a use case if you do not first fully understand the design and architecture that supports the use case? And how will you know that you are implementing that use case at the right time without proper planning?

As the Jolt judges point out, the design, architecture and planning tools we choose to use in an Agile development shop “must be less heavy than in years past and more easily configured to fit the needs of the organization, rather than implementing a specific methodology to which the organization must migrate its orientation, if not its processes.”

I’m excited to check out the winners the judges have chosen. Visit the link at the beginning of this post and check them out yourself.

Whatever tools you choose for doing your design, architecture and planning, do not make the mistake of thinking that these critical phases of development are eliminated by Agile. You may go about these phases with a new attitude toward what is important and with a more realistic view of the realities of the evolution of a product during all phases of development, but you will not ignore them without significant risk. In fact, I think they need to be embraced by Agile teams remaining focused on the use cases and not the frameworks.

C# Basics: On the Importance of the Using Statement and IDisposable

The world of .NET programming is full of objects that implement the IDisposable interface. File, Font, DataContext, Stream, DbConnection, and many more—all implement the IDisposable interface. And for good reason. They touch the outside world and the outside world is messy, full of resource allocations that can only be used one at a time and must be explicitly returned to their owner to be used by another caller.

So in this 8th installment of C# Basics, let’s take a look at how best to use the IDisposable interface by simply using the using statement. The using statement is really just syntactic sugar but it helps you produce much more readable and reliable code.

using (var myobj = new MyDisposable())
{
  myobj.DoSomething();
}

You can even stack them like this:

using (var myobj = new MyDisposable())
using (var otherobj = new MyOtherDisposable())
{
  var x = myobj.DoSomething();
  otherobj.DoAnotherThing(x);
}

Of course, you could write your own resource cleanup explicitly, but it can start to look a bit messy and as lazy as we often are, the cleanup often gets forgotten.

var myobj = new MyDisposable();
var otherobj = new MyOtherDisposable();
try
{
  var x = myobj.DoSomething();
  otherobj.DoAnotherThing(x);
}
finally
{
  if (otherobj != null) otherobj.Dispose();
  if (myobj != null) myobj.Dispose();
}

Use the using statement where you can. It's not a one size fits all syntax sugar, but it is sweet where it fits.

Dennis Ritchie, RIP, Thanks for C

Dennis Ritchie, the Father of C, passed away a few days ago. A friend asked me today, “Would we have had a Steve Jobs without first having a Dennis Ritchie?” He made a great point. The most common programming language for the Apple platform today is Objective-C. Mr. Ritchie was not an attention hound it seems. I doubt more than 1% of the people who know who Steve Jobs is would know who Dennis Ritchie is, and yet, we in this business of programming owe him a great debt of gratitude.

I know there are C detractors, but the ubiquity of C and all it’s descendants cannot be denied. Andrew Binstock, in his column today on Dr. Dobbs, quoted Mr. Ritchie as having said:

“C is quirky, flawed, and an enormous success. [And UNIX] is very simple, it just needs a genius to understand its simplicity.” ~Dennis Ritchie

Oh yeah, and I forgot to mention he was also deeply involved in developing UNIX. And how many derivatives of UNIX are there? And how many other operating systems at their core are so like UNIX as to make the grand old operating system the true grandfather or at least uncle of every seriously utilized operating system in the world?

I will end by borrowing Mr. Binstock’s conclusion:

“Ritchie saw in language what others could not see, in operating system what others had not built, and in the world around him what others did not realize. His insight and the elegance of his work will be missed by all programmers, even in future generations who, as he would want it, might know nothing of him.” ~Andrew Binstock

Thanks, Dennis Ritchie. Thanks for C. And thanks for UNIX.

And whatever you do, don’t miss this farewell from MuppetLabs. Thanks for sharing it, Dave.

C# 5.0 Exciting New Asynchrony Features Coming

I’m gradually catching up with presentations from last month’s BUILD conference on Channel 9. Today I ran into a blog post by Samuel Jack in the UK on the topic of what’s new in C# 5.0. I recommend you read his blog, but more strongly recommend you watch Anders Hejlberg’s presentation on Channel 9 about C# 5.0 and VB 11.0.

Among my favorite goodies are the async and await keywords. For anyone who has put off learning how to write asynchronous application behavior because it is just too hard to understand, you are in luck. Wait a few more months, perhaps a year, and you can learn just a few easy concepts about how to use async and await keywords in your code and you will be an asynchronous coding wizard.

Exciting times!

C# Basics: const vs readonly vs static

In this 7th installment of C# Basics, I’m going to cover the differences between the modifiers const, readonly, and static, specifically in reference to class members where the developer wishes to use the fields or properties in client code that cannot change the value of that member.

To illustrate and discuss the differences between each modifier, I’ve put together a contrived set of classes with the least amount of code I can think to add in order to review and to examine the differences in the IL (intermediate language) taken from Ildasm.exe. (Visit this nice page to learn how to use Ildasm.exe from with Visual Studio.)

using System;

namespace ReadOnlyConstant
{
  public class MyConstOnly
  {
    // assigned at declaration and used at compile time only
    public const int Age = 20;
  }

  public class MyReadOnly
  {
    // can only be assigned in declaration or constructor
    public readonly int Age = 0;
    public MyReadOnly()
    {
      Age = 20;
    }
  }

  public static class MyStaticOnly
  {
    // can be assigned within class but "readonly" for client
    private static int _age = 20;
    public static int Age { get { return _age; } }
  }
}

The goal in each class above is to create a public field or member that can be read by client code but cannot be changed by client code. Looking through the IL produced by compiling this code can also be instructive even if you do not fully understand each and every IL instruction. Take a look here at these classes under the compiled covers and notice that the MyConstOnly class does not have a getter method to retrieve Age nor is it's value set in the .ctor but only noted by the compiler in the .field definition for use by the compiler later should client code use it. Then read through to the client code and see its IL code as well.

// =============== CLASS MEMBERS DECLARATION ===================

.class public auto ansi beforefieldinit ReadOnlyConstant.MyConstOnly
       extends [mscorlib]System.Object
{
  .field public static literal int32 Age = int32(0x00000014)
  .method public hidebysig specialname rtspecialname 
          instance void  .ctor() cil managed
  {
    // Code size       7 (0x7)
    .maxstack  8
    IL_0000:  ldarg.0
    IL_0001:  call       instance void [mscorlib]System.Object::.ctor()
    IL_0006:  ret
  } // end of method MyConstOnly::.ctor

} // end of class ReadOnlyConstant.MyConstOnly

.class public auto ansi beforefieldinit ReadOnlyConstant.MyReadOnly
       extends [mscorlib]System.Object
{
  .field public initonly int32 Age
  .method public hidebysig specialname rtspecialname 
          instance void  .ctor() cil managed
  {
    // Code size       25 (0x19)
    .maxstack  8
    IL_0000:  ldarg.0
    IL_0001:  ldc.i4.0
    IL_0002:  stfld      int32 ReadOnlyConstant.MyReadOnly::Age
    IL_0007:  ldarg.0
    IL_0008:  call       instance void [mscorlib]System.Object::.ctor()
    IL_000d:  nop
    IL_000e:  nop
    IL_000f:  ldarg.0
    IL_0010:  ldc.i4.s   20
    IL_0012:  stfld      int32 ReadOnlyConstant.MyReadOnly::Age
    IL_0017:  nop
    IL_0018:  ret
  } // end of method MyReadOnly::.ctor

} // end of class ReadOnlyConstant.MyReadOnly

.class public abstract auto ansi sealed beforefieldinit ReadOnlyConstant.MyStaticOnly
       extends [mscorlib]System.Object
{
  .field private static int32 _age
  .method public hidebysig specialname static 
          int32  get_Age() cil managed
  {
    // Code size       11 (0xb)
    .maxstack  1
    .locals init ([0] int32 CS$1$0000)
    IL_0000:  nop
    IL_0001:  ldsfld     int32 ReadOnlyConstant.MyStaticOnly::_age
    IL_0006:  stloc.0
    IL_0007:  br.s       IL_0009

    IL_0009:  ldloc.0
    IL_000a:  ret
  } // end of method MyStaticOnly::get_Age

  .method private hidebysig specialname rtspecialname static 
          void  .cctor() cil managed
  {
    // Code size       8 (0x8)
    .maxstack  8
    IL_0000:  ldc.i4.s   20
    IL_0002:  stsfld     int32 ReadOnlyConstant.MyStaticOnly::_age
    IL_0007:  ret
  } // end of method MyStaticOnly::.cctor

  .property int32 Age()
  {
    .get int32 ReadOnlyConstant.MyStaticOnly::get_Age()
  } // end of property MyStaticOnly::Age
} // end of class ReadOnlyConstant.MyStaticOnly

// =============================================================

You can read the MSDN full explanations of each modifier but here’s the basics:

const
Can only be assigned a value in declaration and can only be a value type or string. Use the const modifier when you KNOW the value won’t change. If you think it might change at a later date and your assembly is distributed as a compiled library, consider one of the other modifiers to assure that you don’t have a value you didn’t expect in your client code. (See use code sample below.)

readonly
Can assign a value at declaration or in the class constructor. It is important to note that if you use a reference type with modifiable members, your client code can still modify those members even if it cannot assign a value to the readonly reference. Note in the IL above that the initialization of the declared value occurs in the .ctor before the assignment in the .ctor, so if you are wondering which would be better, now you have some insight into that question.

static
Can assign the value of the private member anywhere within the class code. Note the initialization of the value in the static .ctor of the class. You could also assign the value in some other method later but with the public property implementing only a get, the client code cannot assign a value.

And here is the client code and it’s IL just below it. The most important point to note in the IL is that the client code is compiled with the const’s literal value, NOT a get to the class. This is why you must watch for the use of a const that could change with a new library. Make sure you compile your client code against that new library when you get it or you could be very sorry when the library is using one const compiled value and you’re using another.

namespace TestConsole
{
  class Program
  {
    static void Main(string[] args)
    {
      // compiler will replace with constant value
      // If the referenced assembly is changed to 40 and this is
      // not compiled again against that new assembly, the value
      // for mcoAge will still be 20. (See IL below.)
      int mcoAge = MyConstOnly.Age;

      MyReadOnly mro = new MyReadOnly();
      int mroAge = mro.Age;

      int msoAge = MyStaticOnly.Age;

      Console.WriteLine("{0} {1} {2}", mcoAge, mroAge, msoAge);
    }
  }
}

// output: 20 20 20

// and here is the IL with some of my own comments

// =============== CLASS MEMBERS DECLARATION ===================

.class private auto ansi beforefieldinit TestConsole.Program
       extends [mscorlib]System.Object
{
  .method private hidebysig static void  Main(string[] args) cil managed
  {
    .entrypoint
    // Code size       53 (0x35)
    .maxstack  4
    .locals init ([0] int32 mcoAge,
             [1] class [ReadOnlyConstant]ReadOnlyConstant.MyReadOnly mro,
             [2] int32 mroAge,
             [3] int32 msoAge)
    IL_0000:  nop
    IL_0001:  ldc.i4.s   20   //NOTE: literal value assigned - no mention of MyConstOnly class
    IL_0003:  stloc.0
    IL_0004:  newobj     instance void [ReadOnlyConstant]ReadOnlyConstant.MyReadOnly::.ctor()
    IL_0009:  stloc.1
    IL_000a:  ldloc.1
    IL_000b:  ldfld      int32 [ReadOnlyConstant]ReadOnlyConstant.MyReadOnly::Age
    IL_0010:  stloc.2
    IL_0011:  call       int32 [ReadOnlyConstant]ReadOnlyConstant.MyStaticOnly::get_Age()
    IL_0016:  stloc.3
    IL_0017:  ldstr      "{0} {1} {2}"
    IL_001c:  ldloc.0
    IL_001d:  box        [mscorlib]System.Int32
    IL_0022:  ldloc.2
    IL_0023:  box        [mscorlib]System.Int32
    IL_0028:  ldloc.3
    IL_0029:  box        [mscorlib]System.Int32
    IL_002e:  call       void [mscorlib]System.Console::WriteLine(string,
                                                                  object,
                                                                  object,
                                                                  object)
    IL_0033:  nop
    IL_0034:  ret
  } // end of method Program::Main

  .method public hidebysig specialname rtspecialname 
          instance void  .ctor() cil managed
  {
    // Code size       7 (0x7)
    .maxstack  8
    IL_0000:  ldarg.0
    IL_0001:  call       instance void [mscorlib]System.Object::.ctor()
    IL_0006:  ret
  } // end of method Program::.ctor

} // end of class TestConsole.Program

// =============================================================

Meaningless Meetings Meander Morosely Mostly

Recently I came across an interesting list of top ten time wasters at work. I jotted them down but have lost the source, so I apologize for not providing the link. Here’s their list:

  1. Instant Messaging
  2. Over-Reliance on Email
  3. Meandering Meetings
  4. Short Gaps Between Meetings
  5. Reacting to Interruptions
  6. Ineffective Multi-Tasking
  7. Disorganized Workspace
  8. Personal Communication
  9. Web Surfing "Breaks"
  10. Cigarette/Coffee Breaks

It’s a decent list that one might find on many top ten sites, I’m sure. But it’s not terribly accurate. You see, the biggest time waster in the enterprise is the Meaningless, Meandering Meeting Machine. So here’s my revised list:

  1. Thinking about meetings.
  2. Planning meetings.
  3. Doodling in meetings.
  4. Talking about meetings.
  5. Scheduling and rescheduling meetings.
  6. Meetings.
  7. Follow-up meetings.
  8. Looking or asking for minutes from meetings.
  9. Assuming people will do what was agreed to in meetings.
  10. Making lists about meetings.

I have seen good, productive employees become consumed by the Meaningless Meetings Machine. It is an endless recursive function leading to enterprise stack overflow. Sometimes executives are lost for months in the Machine and when employees smart enough or lucky enough to stay away from the Machine are asked if they have seen Mr. Soandso, they say, No, and hurriedly move on lest they too be sucked into the Machine.

Seek to Establish a Corporate Culture of Honesty

In April of this year, an executive of another company told me in a telephone conversation, "Everybody lies. You just have to get used to it."

I was stunned by the statement and have given it a lot of thought since then. It reminded me of a scene from Babylon 5 that went something like this:

Garibaldi: Everybody lies.
Edgars: That's a very sad view of the universe, Mr. Garibaldi.
Garibaldi: Yeah, well, it's the only one I got. And it works for me.
Edgars: The truth will be revealed in a couple of days. How many people can say that?
Garibaldi: I don't know. But I think the last guy got thirty pieces of silver for the same job.

Dishonesty in this technological age has even produced its own vocabulary, specifically the verb "to blag." The third definition of "blag" in the Urban Dictionary:

To convince another person that all the stuff you just made up is in fact true and worthy. Example: "Caught in a tight spot, Harry blagged his way through the conversation and somehow got the job."

I do not wish to accept the dim view that everybody lies. And I certainly do not wish to get used to it, in any way shape or form. And yet there is growing evidence both anecdotally in my own professional career and globally that the enterprise often creates a dishonest culture. Author Brian Amble in Management Issues filed a piece in 2005 entitled Corporate Culture Encourages Lying. The author mentions "blagging" and makes these three very interesting points:

"The rot starts at the top, revealing a surprisingly ambivalent attitude amongst...bosses towards the honesty – or otherwise - of their staff."

"The vast majority of company directors and senior managers believe it is wrong for their employees to lie to them. But almost half are comfortable with those same employees telling untruths on their behalf to their customers – with female bosses even more tolerant of this sort of behavior than their male colleagues."

"[Microsoft] encourages decision-making [using a] technology-based process that creates permanent digital records and maintains the integrity of the information on which those decisions are based."

In my experience blagging is most especially prevalent in the world of software development where management clings to the notion, and consultants sell the latest process magic to reaffirm the assumption, that building software is like building a house, sufficiently predictable and repeatable so as to accurately establish a firm budget, timeline and resource commitment before ground is broken. Software artisans then find themselves in the very uncomfortable position of feeling pressured to lie to their patron bosses who clearly do not wish to know that building software is often as much an unpredictable art as it is in any way a predictable science.

Software craftsmen who boldly stand their ground and declare, "I don't know how long it will take to build that," are more often attacked and labeled malcontent and misfit, shunned by managers who insist that they can bend reality to their will while they ignore what their experts have told them. And lower management who know better are caught in the ultimate catch 22. They may believe their lead developers or software architects when they say, "we cannot deliver all these features on that date," but they cannot report that fact to their own managers or customers, so they choose to lie in order to protect and even advance their own career knowing that if the implementation team fails, they may avoid accountability by throwing the developers under the bus, asserting the implementation team is incompetent and ought to be replaced.

Given the impossible circumstances in which software craftsmen find themselves, they often resort to lying to themselves and their bosses asserting that they can produce some piece of software, despite the general lack of specific requirements and understanding of the problem domain or the target users, within the time budget and resource constraints imposed by management. And then in order to avoid being exposed as just another blagger, the software artisan (or collectively the team) works impossible hours, makes imprudent quality compromises and desperately seeks for external circumstances which can be blamed for delays. Knowing that such responsibility diverting opportunities always arise, the software artisan begins to accept and even embrace the culture of lies produced by her circumstances until she does not even recognize the truth.

I know that this scenario is not universal. There are organizations who take great steps, incorporating technology and constant vigilance, to avoid the traps of dishonesty. These organizations are led by honest people who genuinely listen to their people and avoid the trap of boxing their software craftsmen into a set of assumptions that are untenable at best and downright foolish at worst. Dr. Rhonne Sanderson said it best in an article by Marcia A. Reed-Woodard entitled Don't lie to me: dishonesty can ruin professional and personal relationships:

"Although lying provides an easy out in the short-term, it comes with serious repercussions," says Dr Rhonne Sanderson, a Dallas--Fort Worth area licensed psychotherapist. He maintains that the fallout from lying can hurt others, ruin relationships, as well as rob the liar of integrity, credibility, confidence, and self-esteem. "Lying only exacerbates the real problem."

If we let honesty govern our dealings with our fellow man, we will all be the better for it. We do not need to sacrifice civility for honesty, nor should we mistake an honest disagreement for disrespect or insubordination, for when we do, we encourage others to be dishonest through their silence.

We can boldly speak the truth and expect others to do so. We can hold ourselves and others to account. We need not settle for the notion that everybody lies. And we certainly do not need to get used to it. We can and ought to do and be better.

C# Thread Safe Class Essentials

In this post I'm recording my own code sample for future reference. It is not meant to be perfectly usable or even a representation of best practices but a practical reminder note of the most common thread safe code I end up writing. The code is a thread safe singleton class that demonstrates the use of the lock keyword, the Interlocked class, ReadWriterLockSlim class, and one of the new concurrent collections available in .NET 4.0.

In the code below, I've only shown the ConcurrentDictionary being used. There is a gold mine of power in the other concurrent types to be explored in other posts.

using System;
using System.Collections.Generic;
using System.Collections.Concurrent;
using System.Threading;

namespace SafeThreads
{
  public sealed class DataCache
  {
    // Thread Safe Singleton Pattern
    //+++++++++++++++++++++++++++++++++++++++++++++++++++++
    
    //volatile to insure assignment before read access
    private static volatile DataCache _instance;
    
    //the sync root object - clever name
    private static object _threadBolt = new object();

    private DataCache() { } //cannot create outside this

    public static DataCache Instance
    {
      get
      {
        if (null == _instance)
        {
          lock (_threadBolt)
          {
            if (null == _instance)
            {
              _instance = new DataCache();
            }
          }
        }
        return _instance;
      }
    }
    

    // Thread Safe with lock
    //+++++++++++++++++++++++++++++++++++++++++++++++++++++

    private long _callCount = 0;
    private long _updateCount = 0;

    public long UpdateCountWithLock(long callCount)
    {
      lock (_threadBolt)
      {
        _callCount += callCount;
        _updateCount++;
        return _updateCount;
      }
    }

    // Thread Safe Interlocked class
    //+++++++++++++++++++++++++++++++++++++++++++++++++++++

    public long UpdateCountWithInterlocked(long callCount)
    {
      Interlocked.Add(ref _callCount, callCount);
      Interlocked.Increment(ref _updateCount);
      return _updateCount;
    }

    private Tuple<int, string> _currentItem = 
      new Tuple<int, string>(0, string.Empty);

    public Tuple<int, string> CurrentItemInterlocked
    {
      get { return _currentItem; }
      set
      {
        Interlocked.Exchange<Tuple<int, string>>(ref _currentItem, value);
      }
    }


    // Thread Safe ReaderWriterLockSlim
    //+++++++++++++++++++++++++++++++++++++++++++++++++++++

    private ReaderWriterLockSlim _slimLock = new ReaderWriterLockSlim();
    private Dictionary<string, int> _keys = new Dictionary<string, int>();
    private List<string> _values = new List<string>();

    public int Count
    {
      get
      {
        _slimLock.EnterReadLock();
        try
        {
          return _values.Count;
        }
        finally
        {
          _slimLock.ExitReadLock();
        }
      }
    }

    public string this[int index]
    {
      get
      {
        _slimLock.EnterReadLock();
        try
        {
          if (_values.Count > index && index > -1)
            return _values[index];
          else
            return null;
        }
        finally
        {
          _slimLock.ExitReadLock();
        }
      }
    }

    public string this[string key]
    {
      get
      {
        _slimLock.EnterReadLock();
        try
        {
          if (_keys.ContainsKey(key))
          {
            return _values[_keys[key]];
          }
          else
            return null;
        }
        finally
        {
          _slimLock.ExitReadLock();
        }
      }

      set
      {
        _slimLock.EnterWriteLock();
        try
        {
          if (_keys.ContainsKey(key))
          {
            _values[_keys[key]] = value;
          }
          else
          {
            _values.Add(value);
            _keys.Add(value, _values.Count - 1);
          }
        }
        finally
        {
          _slimLock.ExitWriteLock();
        }
      }
    }

    //demonstrate upgrade from read to write lock
    public AddUpdateResult AddOrUpdate(string key, string value)
    {
      AddUpdateResult result = AddUpdateResult.Unchanged;
      _slimLock.EnterUpgradeableReadLock();
      try
      {
        if (_keys.ContainsKey(key))
        {
          if (value == _values[_keys[key]])
            return result;
          else
            result = AddUpdateResult.Updated;
        }
        else
          result = AddUpdateResult.Added;

        //upgrade to write lock
        _slimLock.EnterWriteLock();
        try
        {
          if (result == AddUpdateResult.Updated)
          {
            _values[_keys[key]] = value;
          }
          else
          {
            _values.Add(value);
            _keys.Add(value, _values.Count - 1);
          }
        }
        finally
        {
          _slimLock.ExitWriteLock();
        }
        return result;
      }
      finally
      {
        _slimLock.ExitUpgradeableReadLock();
      }
    }

    public enum AddUpdateResult
    {
      Added,
      Updated,
      Unchanged
    };


    // Thread Safe Collections
    //+++++++++++++++++++++++++++++++++++++++++++++++++++++

    private ConcurrentDictionary<string, string> _concurrentCache = 
      new ConcurrentDictionary<string, string>();

    public ConcurrentDictionary<string, string> ConcurrentCache
    {
      get { return _concurrentCache; }
    }
  }
}