tsJensen

A quest for software excellence...

Complexity is the Enemy of Success

Whether it be in code or business deals, complexity kills. If you're an architect and you love the elegance of endless inheritance where everything is a descendant of MyCoolRootObject or an venture capitalist trying to tie off every risk with carefully structured language that leaves a founder in the lurch and you in the driver's seat of the getaway car, you are the enemy of success. If you flout the team's style guild and name your class members with freaky names and patterns only you can recognize, you are an enemy of success. If you're a framework developer and you believe you have to add every possible toy feature in the universe to your framework, you are an enemy of success.

And enemies of success lose! Lovers of complexity may win a battle here or there, but the ash heap of history is full of them. Consider any number of technologies that have become so overbloated and difficult to work with that developers and architects look for simpler solutions. Examine the many thousands of failed startups killed by pencil pushing pinheads with no other agenda than to make the deal difficult in hopes of making it perfect, only to kill the deal with a stulted obsession with detail and gaining the advantage in every paragraph.

Simplicity is the key to success.

SqlParameter Factory

Often in my line of work, I need to create parameterized queries in code. All those overloaded constructors. Yuck. So, like a lazy guy would, I built a factory. I use it everywhere. In my personal projects and even at work when no one is looking. Here it is:

static SqlParameter CreateParameter(string name, SqlDbType type,
    ParameterDirection direction, int? size, byte? precision, object value)
{
    SqlParameter param = new SqlParameter(name, type);
    param.Direction = direction;
    if (size.HasValue) param.Size = size.Value;
    if (precision.HasValue) param.Precision = precision.Value;
    if (value != null) param.Value = value; else param.Value = DBNull.Value;
    return param;
}

And then when I'm building a command, I just do this:

cmd.Parameters.Add(DataAccess.CreateParameter("@RETURN_VALUE",
    SqlDbType.Int, ParameterDirection.ReturnValue, null, null, null));
cmd.Parameters.Add(DataAccess.CreateParameter("@ID",
    SqlDbType.Int, ParameterDirection.Input, null, null, id));
cmd.Parameters.Add(DataAccess.CreateParameter("@NAME",
    SqlDbType.VarChar, ParameterDirection.Input, 50, null, name));

Of course, it won't work for everything, but it makes quick work out of most of my parameter creation. Hooray for the factory.

Process Magic Booch Style

I recently picked up Booch, Jacobsen and Rumbaugh's new book Object Oriented Analysis and Design with Applications. It's like the textbook I never had in the OO classes I never took. (Yeah, I'm just another self taught bozo who knows the difference between a five minute class exercise and a multiple month enterprise development project.)

Among others, I have especially enjoyed Chapter 6 called simply Process. Let me quote the opening paragraph and you'll know why.

"The amateur software engineer is always in search of magic, some sensational method or tool whose application promises to render software development trivial. It is the mark of the professional software engineer to know that no such panacea exists. Amateurs often want to follow cook-book steps; professionals know that such approches to development usually lead to inept design products, born of a progression of lies, and behind which developers can shield themselves from accepting responsibility for earlier misguided decisions." pg.247

There's more and the book is a treasure trove of wisdom, but when I read this paragraph, it was nice to feel as if I had met the estimeed Booch, Jacobsen and Rumbaugh definition of professional software engineer.

I do remember looking for magic bullets when I first started teaching myself how to do software development. Fortunately through long trial and error and even greater opportunities to learn from true professionals, I gave up on looking for magic solutions long ago. Since then my life has been easier and busier and much more rewarding with regard to software development.

To their credit, the authors in Chapter 6 review the strengths and weaknesses of both agile and more traditional plan driven (sometimes called waterfall) process approaches. They begin their thesis in this chapter in addressing the traits of a successful project. Now having this is true magic. Here they are:

  • "Existence of a strong architectural vision."
  • "Application of a well managed iterative and incremental development lifecycle."

The killer qualifiers in those two statements are: "strong" and "well managed." Yikes! These are qualifications that are difficult to come by. When you do, grab them up and hold on to them for dear life. I think most of us can agree that "weak" and "poorly managed" will result in disaster every time.

Requirements in Text or Why I Hate Microsoft Word and Excel

In an ever more agile development world, we gradually learn to accept that requirements change constantly. Of course, it has always been thus, but I do remember a time when we pretended that requirements were locked and could not be changed. But then I remember delivering a product some months later which did not then meet the client's requirements despite what they had agreed to only months before. Their requirements changed, you see.

Now I work in an environment where this norm is the norm. Requirements change and they change often. Or better put, requirements are discovered daily and old requirements change into new requirements nearly as often. This is the unavoidable nature of the business I'm in. I accept it.

What I hate and cannot accept in this ever changing world are requirements documents written in a proprietary binary format. Sure they're stored in source control, so when I do a little update on my Subversion client in the morning and see that several requirements documents have changed, I'd like to just do a nice simple DIFF on them and see what's changed. But no. Oh, sure there are probably some diff tools out there I could get, but why should I when we could have just written the requirements in text or even a simple transformable XML rather than the binary gobbledygook in a Word or Excel file.

And can anyone tell me how to "blame" a change in a particular line in a Word doc on a certain author? Oh sure, I could use the gooey sticky messy change tracking--no thanks. Just give me a good text file and an editor that can handle it well.

Is my rant a cry for a product or what? Is there an existing product you can recommend? If so, please tell me. And then maybe we can ban the use of Word and Excel for the production and maintenance of requirements. We can say goodbye to the lack of transparency and traceability. We can say hello to simplicity and accountability. Ah, how would it be.

Logging Logger Exceptions

Sometimes your logger throws an exception while logging an exception. At least it does if you have luck swings like mine from time to time.

Here's how I deal with it.

using System.Diagnostics;

try
{
    //some logging code that can throw an exception
    //but the exception has to be handled and logged
    //somehow
}
catch (Exception e)
{
    try
    {
        //write exception to OS event log
        if (!EventLog.SourceExists("MyAppName"))
            EventLog.CreateEventSource("MyAppName", "Application");
        EventLog.WriteEntry("MyAppName", "Log Exception: " + e.Message);
    }
    catch
    {
        //sorry charlie
    }
}

It's not perfect, but it seems to work in most cases.

This code comes in handy especially in Windows Services where in the .NET Framework 2.0 and up (as near as I can tell), an unhandled exception will result in your service being stopped with little or no evidence as to why.

SQL Server TOP vs MySQL LIMIT

I've recently begun to explore MySQL a bit more, mostly out of curiosity. Primarily I work with SQL Server 2005 in my day job but it seems prudent to know a bit more about some of the other databases servers and it's been a while since I dusted off MySQL.

Happily I found that MySQL AB has released a really great ADO.NET provider for MySQL.

If you have used the SQL Server specific ADO.NET provider, you'll easily make the transition to using this one.

But I digress. The reason for this posting was to permanently remind myself and anyone else making a transition between these two database engines that TOP = LIMIT but not in the same place. That is to say, the TOP keyword is used in SQL Server to limit the number of returned rows just prior to the column specifiers. The LIMIT keyword however is used at the end of the select statement. Thus:

SELECT TOP 1 * FROM MYTABLE WHERE FKID = 3

And for MySQL use:

SELECT * FROM MYTABLE WHERE FKID = 3 LIMIT 1;

Yes, I know. It ought to be obvious, but having a quick reminder here will help to seal this minor difference into my brain.

Value of Schema Driven XML Configuration

If you've written an ASP.NET application, you've already taken advantage of schema driven XML configuration perhaps without even realizing it. You may have even ignored the web.config file if you're just starting out with ASP.NET. Or you may have used it extensively without considering the usefulness of using such a construct in your own applications.

My day job involves writing a complex application that implements a myriad of business rules, nearly all of which non-coders need to be able to modify from time to time without bothering the development team. That means using XML. But not just any old XML. I'm talking about well formed, validated XML using a nice XSD schema file controlled by the development team.

I recommend the reader purchase O'Reilly's book XML Schema by Eric van der Vlist. It's been a most trustworthy companion. And then to make the use of this XML configuration dreamscape even easier, I recommend Christian Weyer's contract first tool. I use this extremely useful tool to generate the XAL (XML access layer) so I don't have to write it by hand.

With these two tools in hand, you can create something like this in Visual Studio:

<?xml version="1.0" encoding="utf-8" ?>
<xs:schema id="watchconfig" targetNamespace="http://example.com/v1001/watchconfig.xsd"
    elementFormDefault="qualified"
    xmlns="http://example.com/v1001/watchconfig.xsd"
    xmlns:xs="http://www.w3.org/2001/XMLSchema">
    <xs:element name="watchconfig">
        <xs:complexType>
            <xs:sequence>
                <xs:element ref="watchgroup" minOccurs="0" maxOccurs="unbounded" />
            </xs:sequence>
        </xs:complexType>
    </xs:element>
    <xs:element name="watchgroup">
        <xs:complexType>
            <xs:sequence>
                <xs:element ref="watcher" minOccurs="0" maxOccurs="unbounded" />
            </xs:sequence>
            <xs:attribute name="id" type="xs:string" use="required" />
            <xs:attribute name="literalsconfigpath" type="xs:string" />
        </xs:complexType>
    </xs:element>
    <xs:element name="watcher">
        <xs:complexType>
            <xs:attribute name="sourcepath" type="xs:string" use="required" />
            <xs:attribute name="wippath" type="xs:string" use="required" />
            <xs:attribute name="outpath" type="xs:string" use="required" />
            <xs:attribute name="ssispath" type="xs:string" use="required" />
            <xs:attribute name="intakemap" type="xs:string" use="required" />
        </xs:complexType>
    </xs:element>
</xs:schema>

And an XML file like this:

<?xml version="1.0" encoding="utf-8" ?>
<watchconfig xmlns="http://example.com/v1001/watchconfig.xsd" xmlns:xs="http://www.w3.org/2001/XMLSchema">
    <watchgroup id="clientname" literalsconfigpath="H:\temp\literals_client">
        <watcher sourcepath="H:\temp\client\watch"
                    wippath="H:\temp\client\wip"
                    outpath="H:\temp\client\out"
                    ssispath="H:\temp\client\ssis"
                    intakemap="D:\Contracts\Code\map.xml" />
    </watchgroup>
</watchconfig>

And the nicely generated code like this:

/// <remarks/>
[System.CodeDom.Compiler.GeneratedCodeAttribute("System.Xml", "2.0.50727.42")]
[System.SerializableAttribute()]
[System.Diagnostics.DebuggerStepThroughAttribute()]
[System.ComponentModel.DesignerCategoryAttribute("code")]
[System.Xml.Serialization.XmlTypeAttribute(AnonymousType=true, TypeName="watchconfig")]
[System.Xml.Serialization.XmlRootAttribute(Namespace="http://example.com/v1001/watchconfig.xsd", IsNullable=false, ElementName="watchconfig")]
[System.ComponentModel.TypeConverterAttribute(typeof(System.ComponentModel.ExpandableObjectConverter))]
public partial class Watchconfig
{

/// <remarks/>
private System.Collections.Generic.List<Watchgroup> watchgroup;

public Watchconfig()
{
}

public Watchconfig(System.Collections.Generic.List<Watchgroup> watchgroup)
{
this.watchgroup = watchgroup;
}

[System.Xml.Serialization.XmlElementAttribute("watchgroup", Order=0)]
public System.Collections.Generic.List<Watchgroup> Watchgroup
{
get
{
return this.watchgroup;
}
set
{
if ((this.watchgroup != value))
{
this.watchgroup = value;
}
}
}
}

/// <remarks/>
[System.CodeDom.Compiler.GeneratedCodeAttribute("System.Xml", "2.0.50727.42")]
[System.SerializableAttribute()]
[System.Diagnostics.DebuggerStepThroughAttribute()]
[System.ComponentModel.DesignerCategoryAttribute("code")]
[System.Xml.Serialization.XmlTypeAttribute(AnonymousType=true, TypeName="watchgroup")]
[System.Xml.Serialization.XmlRootAttribute(Namespace="http://example.com/v1001/watchconfig.xsd", IsNullable=false, ElementName="watchgroup")]
[System.ComponentModel.TypeConverterAttribute(typeof(System.ComponentModel.ExpandableObjectConverter))]
public partial class Watchgroup
{

/// <remarks/>
private System.Collections.Generic.List<Watcher> watcher;

/// <remarks/>
private string id;

/// <remarks/>
private string literalsconfigpath;

public Watchgroup()
{
}

public Watchgroup(System.Collections.Generic.List<Watcher> watcher, string id, string literalsconfigpath)
{
this.watcher = watcher;
this.id = id;
this.literalsconfigpath = literalsconfigpath;
}

[System.Xml.Serialization.XmlElementAttribute("watcher", Order=0)]
public System.Collections.Generic.List<Watcher> Watcher
{
get
{
return this.watcher;
}
set
{
if ((this.watcher != value))
{
this.watcher = value;
}
}
}

[System.Xml.Serialization.XmlAttributeAttribute(AttributeName="id")]
public string Id
{
get
{
return this.id;
}
set
{
if ((this.id != value))
{
this.id = value;
}
}
}

[System.Xml.Serialization.XmlAttributeAttribute(AttributeName="literalsconfigpath")]
public string Literalsconfigpath
{
get
{
return this.literalsconfigpath;
}
set
{
if ((this.literalsconfigpath != value))
{
this.literalsconfigpath = value;
}
}
}
}

/// <remarks/>
[System.CodeDom.Compiler.GeneratedCodeAttribute("System.Xml", "2.0.50727.42")]
[System.SerializableAttribute()]
[System.Diagnostics.DebuggerStepThroughAttribute()]
[System.ComponentModel.DesignerCategoryAttribute("code")]
[System.Xml.Serialization.XmlTypeAttribute(AnonymousType=true, TypeName="watcher")]
[System.Xml.Serialization.XmlRootAttribute(Namespace="http://example.com/v1001/watchconfig.xsd", IsNullable=false, ElementName="watcher")]
[System.ComponentModel.TypeConverterAttribute(typeof(System.ComponentModel.ExpandableObjectConverter))]
public partial class Watcher
{

/// <remarks/>
private string sourcepath;

/// <remarks/>
private string wippath;

/// <remarks/>
private string outpath;

/// <remarks/>
private string ssispath;

/// <remarks/>
private string intakemap;

public Watcher()
{
}

public Watcher(string sourcepath, string wippath, string outpath, string ssispath, string intakemap)
{
this.sourcepath = sourcepath;
this.wippath = wippath;
this.outpath = outpath;
this.ssispath = ssispath;
this.intakemap = intakemap;
}

[System.Xml.Serialization.XmlAttributeAttribute(AttributeName="sourcepath")]
public string Sourcepath
{
get
{
return this.sourcepath;
}
set
{
if ((this.sourcepath != value))
{
this.sourcepath = value;
}
}
}

[System.Xml.Serialization.XmlAttributeAttribute(AttributeName="wippath")]
public string Wippath
{
get
{
return this.wippath;
}
set
{
if ((this.wippath != value))
{
this.wippath = value;
}
}
}

[System.Xml.Serialization.XmlAttributeAttribute(AttributeName="outpath")]
public string Outpath
{
get
{
return this.outpath;
}
set
{
if ((this.outpath != value))
{
this.outpath = value;
}
}
}

[System.Xml.Serialization.XmlAttributeAttribute(AttributeName="ssispath")]
public string Ssispath
{
get
{
return this.ssispath;
}
set
{
if ((this.ssispath != value))
{
this.ssispath = value;
}
}
}

[System.Xml.Serialization.XmlAttributeAttribute(AttributeName="intakemap")]
public string Intakemap
{
get
{
return this.intakemap;
}
set
{
if ((this.intakemap != value))
{
this.intakemap = value;
}
}
}
}

(Sorry, indentation got hosed, but you won't need it since you'll be generating this code with one click.)

 

 

 

 

The Case Against Use Cases

I'm currently working on a rather complex ETL system in the medical industry. There are new business rules uncovered daily as development and analysis proceeds in parallel due to overwhelming business urgencies. The use cases in the system are limited pretty much to "Process File." Everything else is buried in a system of business rules more complex than I care to think about very often and a series of events and event handlers which process the file and implement the business rules.

My own approach to this challenge could have been much more organized had I purchased Karl E. Weigers's book More About Software Requirements earlier on in this project. He says, "Use cases are less valuable for projects involving data warehouses, batch processes, hardware projects with embedded control software, and computationally intensive applications. In these softs of systems, the deep complexity doesn't lie in the user-system interactions. It might be might be worthwhile to identify use cases for such a product, but use case analysis will fall short as a technique for defining all the system's behavior."

I could not agree more. Weigers goes on to recommend the use of event-response tables to provide a way of documenting the requirements of such complex systems which have little if any interaction with users. Granted, you could write a use case using the machine or file system or OS or scheduler or some other non-human entity as the actor, but the analogies break down when trying to document the requirements of the complex rules within the case.

The event-response table is a simple approach to organizing these details that in fact works much better than an ad hoc method of writing it all down in sequential paragraphs and then asking developers, in this case that developer being me, to interpret those requirements and design and code a solution that really works.

You simply need a three column table with the following headers: Event, System State, and Response.

Breaking up functional requirements in a complex rules-driven system with minimal human interaction can be a daunting task. You can make it a bit easier by using some simple organizing structures such as the event-response table.