Enterprise Content Management for the Win in 2012

December was an interesting month with multiple pieces on the chess board moving all at once insuring an interesting and exciting new year for me. And that new year is upon us. I picked up my copy of CIO Insight magazine, the November/December 2011 issue, this morning and read with delight the article entitled Where the IT Dollars Are Headed in 2012.

One passage in the study mentioned in the article (included in the print version and linked to online) caught my eye:

“Elsewhere in enterprise applications, collaboration, and content and project management are rapidly gaining popularity, which makes sense given the new mobile opportunities for these categories. Collaboration and content management are seeing increases in 2012 over 2011 of roughly 10 percentage points in the number of organizations budgeting; and among those that already were spending in these areas, budget growths are averaging 14.6 percent and 7.3 percent, respectively—both significantly higher than 2011’s growth.”

I have a recently renewed and very keen interest in the potential growth of the enterprise content management market. And I will blog more about this in the coming weeks which are sure to open up new opportunities for personal and professional growth.

Estimating Software Features and Fixed Feature Compensation from 8th Light

I just finishing reading a fascinating blog post by Paul Pagel of 8th Light called Fixed Feature. I recommend that you read the post and download and study the linked PDF entitled From Estimate to Commitment.

Companies willing to pay for software development on a time-and-materials basis are fewer and farther between. They don’t have the resources to monitor such work to assure themselves that the team is progressing as it should. Understandably, they cannot shoulder this risk. So they want their teams, internal and external, to give them a fixed bid for a project: fixed time, fixed cost, fixed features.

Pagel points out, “software is a craft with a close problem finding to problem solving loop.” In other words, we’re not always very good at anticipating problems in a development project until they are just about to happen. Sometimes we don’t even see them coming. We can minimize these events through continuous review and refinement of our processes and practices, study and evaluation of past performance versus estimates and the assumptions that went into those. Too often we do not take the time to do so, dooming ourselves to historical repetition.

Rather than attempting to created a fixed bid for an entire project, 8th Light, according to Pagel, estimates each feature using PERT and creates a price point for each feature. At the end of an iteration, they bill the client for features accepted by the client. This is an interesting approach to combining the desire for fixed bid pricing and Agile development.

Of course, this or any variation on fixed bid pricing requires that we improve our track record on creating estimates. If you visit our friends in Mountain View and search for estimating software development, you will find over four million results. There are tools and whitepapers and blog posts galore. But before you spend more money on tools or consultants, look to your own experience. Be willing to record what you’ve done, evaluate it honestly and boldly, and make adjustments based on what you have learned. Surely this will be less painful than blindly repeating the past.

Read Pagel’s work on this subject and let me know what you think.

Custom Recursive Model Validation in .NET Using Data Annotations

Need to write your own model validator outside the scope of an application framework such as ASP.NET MVC? A short while ago, I needed to do just that. I was writing a WCF service with a relatively complex data model which required a much greater level of validation than the DataMember attribute’s IsRequired property could provide.

Here’s the solution I found. But first a bit of background.

Validation in ASP.NET MVC 
I’ve been using Data Annotations attributes for view model validation for years in the context of ASP.NET MVC for model validations, client and server side. I always took the server side validation for granted and looked at the client side with greater interest. When getting started with ASP.NET MVC, I used Steve Sanderson’s xVal library. Then I switched to ASP.NET MVC 3’s client side validation.

For client side validation, I’m really starting to like jQuery’s unobtrusive validation and the ASP.NET MVC’s HtmlHelper class and its GetUnobtrusiveValidationAttributes method. And for the server side, view model binding and the ModelState.IsValid property works fine.

Validation Without an App Framework 
But this post is not about client side validation or server side validation in ASP.NET MVC. I had to do the validation against the model in the WCF service which, as far as I know, does not have a nice model validation facility other than its own data model serialization which just throws a nasty exception should the data being passed over the wire not provide required data.

So why not use the same approach used by our friends in the ASP.NET MVC world. A little Google investigation turned up something that looked like exactly what I wanted. I found an answer posted to StackOverflow by Mike Reust which included a link to his DataAnnotationsValidatorRecursive library. (Gotta love StackOverflow.)

After some experimentation, I had made some tweaks to Mike’s code and came up with something that worked the way I wanted it to. Here’s a partial example of a data model and how the ValidateObject with it’s optional MinOccursOnEnumerable can be used to require one or more items to be included in a required complex enumerable type that will get recursively.

[DataMember(IsRequired = true), 
	Required(ErrorMessage = "Employer cannot be null."), 
	ValidateObject]
public Employer Employer { get; set; }


[DataMember(IsRequired = true), 
	Required(ErrorMessage = "Children cannot be null."), 
	ValidateObject(MinOccursOnEnumerable = 1)]
public Child[] Children { get; set; }

Using the recursive validation library is easy. Here’s an example:

//perform validation using DataAnnotations for custom validation messages
List<ValidationResult> results = new List<ValidationResult>();
bool isValid = DataAnnotationsValidator.TryValidateObjectRecursive<MyModelData>(model, results);

//now examine isValid and the results

And here’s the code for the validation extensions and ValidateObject attribute.

using System.Collections;
using System.Collections.Generic;
using System.ComponentModel.DataAnnotations;
using System.Linq;
using System;

namespace MyValidator
{
	public static class DataAnnotationsValidator
	{
		public static bool TryValidateObject(object obj, ICollection<ValidationResult> results)
		{
			return Validator.TryValidateObject(obj, new ValidationContext(obj, null, null), results, true);
		}

		public static bool TryValidateObjectRecursive<T>(T obj, List<ValidationResult> results)
		{
			bool result = TryValidateObject(obj, results);

			var properties = obj.GetType().GetProperties().Where(prop => Attribute.IsDefined(prop, typeof(ValidateObjectAttribute)));

			foreach (var property in properties)
			{
				var valAttrib = property.GetCustomAttributes(typeof(ValidateObjectAttribute), true).FirstOrDefault() as ValidateObjectAttribute;
				var value = obj.GetPropertyValue(property.Name);

				if (value == null || valAttrib == null) continue;

				var asEnumerable = value as IEnumerable;
				if (asEnumerable != null)
				{
					List<object> items = new List<object>();
					foreach (var enumObj in asEnumerable) items.Add(enumObj);
					foreach (var enumObj in items)
					{
						result = TryValidateObjectRecursive(enumObj, results) && result;
					}
					if (items.Count < valAttrib.MinOccursOnEnumerable)
					{
						string errorMessage = valAttrib.ErrorMessage ?? "MinOccursOnEnumerable validation failed.";
						results.Add(new ValidationResult(errorMessage));
						result = false;
					}
				}
				else
				{
					result = TryValidateObjectRecursive(value, results) && result;
				}
			}

			return result;
		}
	}

	public static class ObjectExtensions
	{
		public static object GetPropertyValue(this object o, string propertyName)
		{
			object objValue = string.Empty;

			var propertyInfo = o.GetType().GetProperty(propertyName);
			if (propertyInfo != null)
			{
				objValue = propertyInfo.GetValue(o, null);
			}
			return objValue;
		}
	}
}

You need to use the ValidateObject attribute on any complex type you want validated deeply. I found out the hard way that if you try to validate all reference objects, you get nasty results on DateTime properties.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Web;

namespace MyValidator
{
	[AttributeUsage(AttributeTargets.Property | AttributeTargets.Field, Inherited = false, AllowMultiple = false)]
	public class ValidateObjectAttribute : Attribute
	{
		int _minOccurs = 0;
		//marker for object properties that need to be recursively validated

		public ValidateObjectAttribute() { }

		public int MinOccursOnEnumerable { get { return _minOccurs; } set { _minOccurs = value; } }

		public string ErrorMessage { get; set; }
	}
}

Of course you can take off on this or Mike’s original code and create your own validation library for a WCF service, a business logic layer, or whatever you need. Good luck and please update me on your validation adventures.

Where Agile Succeeds

In the interests of keeping the U.S. Postal Service in business, I subscribe to a number free technology periodicals. I’m too cheap to pay for anything more than MSDN Magazine. One of the free rags I like to keep in the reading room is Information Week.

A recent article entitled Where Agile Fails caught my eye. You can get a copy of it online if you want to hand over your contact info. I enjoyed the article and it makes a good general point. The author Charles Babcock draws our attention to the fact that many agile teams overlook operations and the challenges brought about by frequent releases into production.

I was only disappointed in the title. Any reader who quickly browses the article, allowing the title to influence conclusions, may walk away with the wrong idea. The title would be more informative if less typographically attractive as Where Agile Teams Fail to Involve Operations, Agile Teams Fail.

When agile development is done right, it includes as part of release and sprint planning some elements of deployment, operations, support and maintenance. Some sprints or releases planned may not go into production, so operations need not be involved too heavily, but I think they should be be apprised of the team’s plans and progress and given the opportunity to prepare in advance for the needs of the application and users.

When a release or sprint will go into production, the operations team should be represented in the planning meetings. This will give them the opportunity to learn what is expected of them and plan accordingly. It will also give them a chance to ask questions, raise concerns or even point out why they will not be able to support a go-live date for a release. And when that is the case, you want to know sooner rather than later.

Where agile succeeds is in the notion that people and interactions are more important than processes. The Manifesto does not indicate that it is referring only to developers. Clearly that is not the case. Operations people are people too. Yes, Fred, they really are.

Agile Software Design, Architecture and Planning Tools

Dr. Dobbs published the Jolt Awards for Design, Architecture and Planning Tools today. The Jolt judges make a very important introductory point before discussing the specific winners.

“On large projects, it can be difficult to state requirements, do the design work, and still maintain Agile's orientation towards accepting — even welcoming — new changes from users.

“Those issues notwithstanding, I've noticed that most large or mid-sized projects with mission-critical implications invariably do indeed gather requirements carefully and design the product accordingly. Agile, if it's in use in the organization, typically is most evident in the coding and testing stages.

“Tools that can capture requirements, help illustrate and validate design, and plan the implementation are still very much needed.“

One of the greatest misunderstandings of Agile software development is the assumption that if one thing is valued over another, the thing with lesser value has no value. Let us review the manifesto:

Manifesto for Agile Software Development

We are uncovering better ways of developing
software by doing it and helping others do it.
Through this work we have come to value:

Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan

That is, while there is value in the items on
the right
, we value the items on the left more.

I have added my own emphasis to highlight the point that is so often overlooked by Agile critics and enthusiasts alike. The critic decries Agile for eliminating design, architecture and planning, and yet Agile does nothing of the sort. The enthusiast rushes to write code thinking that Agile is a magic bridge over the need to endure the tedium of design, architecture and planning, and yet, nothing could be further from the truth.

After all, how can you set out to intelligently implement a use case if you do not first fully understand the design and architecture that supports the use case? And how will you know that you are implementing that use case at the right time without proper planning?

As the Jolt judges point out, the design, architecture and planning tools we choose to use in an Agile development shop “must be less heavy than in years past and more easily configured to fit the needs of the organization, rather than implementing a specific methodology to which the organization must migrate its orientation, if not its processes.”

I’m excited to check out the winners the judges have chosen. Visit the link at the beginning of this post and check them out yourself.

Whatever tools you choose for doing your design, architecture and planning, do not make the mistake of thinking that these critical phases of development are eliminated by Agile. You may go about these phases with a new attitude toward what is important and with a more realistic view of the realities of the evolution of a product during all phases of development, but you will not ignore them without significant risk. In fact, I think they need to be embraced by Agile teams remaining focused on the use cases and not the frameworks.

C# Basics: On the Importance of the Using Statement and IDisposable

The world of .NET programming is full of objects that implement the IDisposable interface. File, Font, DataContext, Stream, DbConnection, and many more—all implement the IDisposable interface. And for good reason. They touch the outside world and the outside world is messy, full of resource allocations that can only be used one at a time and must be explicitly returned to their owner to be used by another caller.

So in this 8th installment of C# Basics, let’s take a look at how best to use the IDisposable interface by simply using the using statement. The using statement is really just syntactic sugar but it helps you produce much more readable and reliable code.

using (var myobj = new MyDisposable())
{
  myobj.DoSomething();
}

You can even stack them like this:

using (var myobj = new MyDisposable())
using (var otherobj = new MyOtherDisposable())
{
  var x = myobj.DoSomething();
  otherobj.DoAnotherThing(x);
}

Of course, you could write your own resource cleanup explicitly, but it can start to look a bit messy and as lazy as we often are, the cleanup often gets forgotten.

var myobj = new MyDisposable();
var otherobj = new MyOtherDisposable();
try
{
  var x = myobj.DoSomething();
  otherobj.DoAnotherThing(x);
}
finally
{
  if (otherobj != null) otherobj.Dispose();
  if (myobj != null) myobj.Dispose();
}

Use the using statement where you can. It's not a one size fits all syntax sugar, but it is sweet where it fits.

Dennis Ritchie, RIP, Thanks for C

Dennis Ritchie, the Father of C, passed away a few days ago. A friend asked me today, “Would we have had a Steve Jobs without first having a Dennis Ritchie?” He made a great point. The most common programming language for the Apple platform today is Objective-C. Mr. Ritchie was not an attention hound it seems. I doubt more than 1% of the people who know who Steve Jobs is would know who Dennis Ritchie is, and yet, we in this business of programming owe him a great debt of gratitude.

I know there are C detractors, but the ubiquity of C and all it’s descendants cannot be denied. Andrew Binstock, in his column today on Dr. Dobbs, quoted Mr. Ritchie as having said:

“C is quirky, flawed, and an enormous success. [And UNIX] is very simple, it just needs a genius to understand its simplicity.” ~Dennis Ritchie

Oh yeah, and I forgot to mention he was also deeply involved in developing UNIX. And how many derivatives of UNIX are there? And how many other operating systems at their core are so like UNIX as to make the grand old operating system the true grandfather or at least uncle of every seriously utilized operating system in the world?

I will end by borrowing Mr. Binstock’s conclusion:

“Ritchie saw in language what others could not see, in operating system what others had not built, and in the world around him what others did not realize. His insight and the elegance of his work will be missed by all programmers, even in future generations who, as he would want it, might know nothing of him.” ~Andrew Binstock

Thanks, Dennis Ritchie. Thanks for C. And thanks for UNIX.

And whatever you do, don’t miss this farewell from MuppetLabs. Thanks for sharing it, Dave.

C# 5.0 Exciting New Asynchrony Features Coming

I’m gradually catching up with presentations from last month’s BUILD conference on Channel 9. Today I ran into a blog post by Samuel Jack in the UK on the topic of what’s new in C# 5.0. I recommend you read his blog, but more strongly recommend you watch Anders Hejlberg’s presentation on Channel 9 about C# 5.0 and VB 11.0.

Among my favorite goodies are the async and await keywords. For anyone who has put off learning how to write asynchronous application behavior because it is just too hard to understand, you are in luck. Wait a few more months, perhaps a year, and you can learn just a few easy concepts about how to use async and await keywords in your code and you will be an asynchronous coding wizard.

Exciting times!