tsJensen

A quest for software excellence...

TechEd: ASP.NET and C# Bonanza

Sadly I was unable to attend, but watching a few videos over the weekend is enough to get my juices flowing for what is coming down the pike for .NET and ASP.NET and C#. Here are a few notes and links:

Amazingly cool stuff coming up for .NET:

Next Generation of .NET for Building Applications

  • .NET Native: (22:40) will eventually be available beyond Win 8 RT apps
  • SIMD: (36:00) up to 8x performance improvement in parallel operations on multiple data streams

Future of Visual Basic and C#

  • Roslyn
    • super fast in-memory next generation .NET compiler
    • inline renaming - wow!
    • open source - on codeplex
  • C#
    • primary ctor: public class Point(int h, int w)
    • getter only auto property: public int Height { get; } = h;

Even more amazingly cool stuff coming for ASP.NET:

The Future of .NET on the Server (intro)

The Future of .NET on the Server (deep dive)

ASP.NET vNext

Oh so much to learn and play with. It’s a good time to be alive!

Diversions in the D Programming Language

I am not a systems programmer, meaning I do not write operating system device drivers or file systems or operating system modules, etc, all written in a language that will compile down to raw machine code. I write in C# primarily which is arguably an applications programming language, running in the much loved .NET Common Language Runtime.

The vast majority of systems programming is done in C and C++. And for some reason, C++ has always been a daunting mess of libraries, odd syntax and pointer and memory allocation madness to me. Even setting up an environment to get the right build libraries, the right compiler and linker, etc., have always led me to fits of impatience. And for that reason and many others, I have stuck to C# and applications development.

But every once in a while, I look in on systems programming to see if anyone has really solved the problems I love to hate with respect to C and C++. And for a few years I’ve read a little about the D programming language here and there. A week ago, over the weekend, I decided to give it a try and really see what I could learn.

I have to say, I have been impressed. The D programming language offers a few things that I would dearly love to see in C#.

1. Exception Safety – the scope keyword

void abc() 
{ 
  auto resource = getresource();  // acquire some resource 
  scope(exit) resource.close();   // close the resource 
  doSomeProcessing();             // do processing
}

As C# programmers, we’re used to the try..catch..finally blocks. And we clean up a resource in the finally block. The trouble with that is many lines of code can end up separating your resource acquisition code from your resource cleanup code. Yes, with vigilance and well written tests, this is okay. But wouldn’t it be cool to be able to tell the compiler, “Hey, when I’m done with this thing I just now created, clean it up for me, no matter what code comes after this in this method.” I would love to see the scope keyword added to C#.

2. Concurrency approach

int perThread;
shared int PerProcess;

In C#, when you declare a class level variable, it is automatically shared between threads. You can use the [ThreadStatic] attribute to get a per thread instance of a given value or object. But it then has to be static. With the D programming language, you get thread safety in class variables. To override that safety, you have to explicitly tell the compiler you want the value shared. While I’m not advocating a change to C# in this regard, I would love to have a way to assure that a variable cannot be modified across thread boundaries.

3. Message based threads

import std.concurrency, std.stdio;
void main() {
   auto low = 0, high = 100;
   auto tid = spawn(&writer);
   foreach (i; low .. high) {
      writeln("Main thread: ", i);
      tid.send(thisTid, i);
      enforce(receiveOnly!Tid() == tid);
   }
}

void writer() {
   for (;;) {
      auto msg = receiveOnly!(Tid, int)();
      writeln("Secondary thread: ", msg[1]);
      msg[0].send(thisTid);
   }
}

For me, this is perhaps the coolest part of the D programming language’s base class library which they call Phobos. Note that main spawns a thread calling writer. The loop in main then sends a message to writer and the loop in writer receives the messages and operates on them and then sends a message back to the original thread.

You can learn a lot about D on www.dlang.org and read more about D concurrency on Informit. And if you want to play with D in Visual Studio, hop on over to see VisualD on dsource.org.

Agile Software Design, Architecture and Planning Tools

Dr. Dobbs published the Jolt Awards for Design, Architecture and Planning Tools today. The Jolt judges make a very important introductory point before discussing the specific winners.

“On large projects, it can be difficult to state requirements, do the design work, and still maintain Agile's orientation towards accepting — even welcoming — new changes from users.

“Those issues notwithstanding, I've noticed that most large or mid-sized projects with mission-critical implications invariably do indeed gather requirements carefully and design the product accordingly. Agile, if it's in use in the organization, typically is most evident in the coding and testing stages.

“Tools that can capture requirements, help illustrate and validate design, and plan the implementation are still very much needed.“

One of the greatest misunderstandings of Agile software development is the assumption that if one thing is valued over another, the thing with lesser value has no value. Let us review the manifesto:

Manifesto for Agile Software Development

We are uncovering better ways of developing
software by doing it and helping others do it.
Through this work we have come to value:

Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan

That is, while there is value in the items on
the right
, we value the items on the left more.

I have added my own emphasis to highlight the point that is so often overlooked by Agile critics and enthusiasts alike. The critic decries Agile for eliminating design, architecture and planning, and yet Agile does nothing of the sort. The enthusiast rushes to write code thinking that Agile is a magic bridge over the need to endure the tedium of design, architecture and planning, and yet, nothing could be further from the truth.

After all, how can you set out to intelligently implement a use case if you do not first fully understand the design and architecture that supports the use case? And how will you know that you are implementing that use case at the right time without proper planning?

As the Jolt judges point out, the design, architecture and planning tools we choose to use in an Agile development shop “must be less heavy than in years past and more easily configured to fit the needs of the organization, rather than implementing a specific methodology to which the organization must migrate its orientation, if not its processes.”

I’m excited to check out the winners the judges have chosen. Visit the link at the beginning of this post and check them out yourself.

Whatever tools you choose for doing your design, architecture and planning, do not make the mistake of thinking that these critical phases of development are eliminated by Agile. You may go about these phases with a new attitude toward what is important and with a more realistic view of the realities of the evolution of a product during all phases of development, but you will not ignore them without significant risk. In fact, I think they need to be embraced by Agile teams remaining focused on the use cases and not the frameworks.

How to Get Berkeley DB SQL API into the .NET System.Data.SQLite Provider

My last post covered getting Berkeley DB up and running with .NET. Now it’s time to take it one step further and build the open source System.Data.SQLite ADO.NET library, replacing the SQLite 3 engine with Oracle’s version of the SQLite.Interop C library that gets embedded into the .NET System.Data.SQLite assembly.

In other words, when you complete the steps below, you’ll have a System.Data.SQLite library that can, supposedly, drop into your .NET projects that currently use the ADO.NET library found on the sqlite.org site. The real difference is that instead of SQLite with single threaded writes, or file locking for writes rather, you will be using the latest Berkeley DB storage engine which supports page level locking to allow more writers, assuming the writers are writing to different pages.

For more information on Oracle’s implementation of the SQLite API which they call the Berkeley DB SQL API and to learn where they differ, you should read the whitepaper: Oracle Berkeley DB SQL API vs. SQLite API.

  1. Download the Berkeley DB dbsql-adodotnet-5.2.28.zip source.
  2. Unlock the zip file (Right-click and select Properties and click the Unlock button.)
  3. Extract the contents to an empty directory of your choice.
  4. Open the SQLite.NET.2010.sln solution file.
  5. Choose Release and then build.
  6. Optionally show all files in the System.Data.SQLite.2010 project and open the Assembly.cs file and change the AssemblyProduct string to something like “System.Data.SQLite-BDB” so that it will show up in file Properties Details tab. This is the only way you will know that this assembly has the Berkeley DB interop engine built into rather than the SQLite engine.

If you’re curious, do a diff between the SQLite.Interop project files in the Oracle version and the original sqlite.org version. Clearly very different animals. Now the only thing that is left is to write up a nice little test in C# to compare the two libraries. The subject of a future post.

UPDATE #1 (9/5/2011): Tests today show that this “new” System.SQLite.Data library DOES NOT create a Berkeley DB database. At least as far as I can tell. So while there is a libdb_sql50.dll in the main Berkeley DB build that implements the sqlite3.dll API, there is no way that I can find so far to use that library from C#. More experimentation to come.

UPDATE #2 (9/5/2011): Modify the UnsafeNativeMethods.cs with line 34 as

private const string SQLITE_DLL = “libdb_sql52.dll”;

and then change the solution’s projects conditional compilation symbols to “SQLITE_STANDARD” and then build. This prevents the SQLite interop embedded DLL from being used. Now when you run your test app, you’ll need to copy to the bin directory of your test app the new built DLLs (see previous post):

libdb_sql52.dll
libdb_stl52.dll
libdb52.dll

I’ll post more on my tests later. So far, I’ve got the basics working but when I push the limits, I get some nasty crashes, so I’m skeptical of this scheme to use the SQLite API over the Berkeley DB engine.

How to Get Berkeley DB 5.2.28 and .NET Examples Working in Visual Studio 2010 SP1

Over the years, I’ve looked into using Berkeley DB for my own C# applications because it has a solid reputation as a very fast, reliable key/value database. Every time I’ve walked away disappointed with the .NET API wrapper—until now. My interest was renewed a few days ago when I noticed an item on StackOverflow comparing SQLite performance to Berkeley DB.  Since acquiring Berkeley DB, Oracle has been busy making it better, including adding better support for the .NET development community.

WARNING: read and understand the license terms and conditions for Berkeley DB before you choose to use it.

I began this most recent review by downloading the MSI Windows installer. DO NOT DO THIS! When I tried to compile the VS2010 solutions (see below), I got all kinds of errors. Next I tried downloading the 45MB zip file. This worked like a charm, except the .NET examples projects had a broken reference which was easily fixed. Follow these steps and you’ll be up and running your .NET app using Berkeley DB in no time.

  1. Download db-5.2.28.zip file.
  2. Right click the zip file and select Properties and click the "Unlock" button. This will unlock and make usable all the files in the zip file for your local machine. DO NOT SKIP this step.
  3. Extract the contents of the zip file to a directory, e.g. C:\Code so that you will have a C:\Code\db-5.2.28 folder.
  4. In that db-5.2.28 folder, you will find a build_windows folder. This will be your home for the next steps. Be sure to follow them in order.
  5. Open Berkeley_DB_vs2010.sln
    1. build debug win32 and x64
    2. build release win32 and x64
  6. Open Berkeley_DB_examples_vs2010.sln
    1. build debug win32 and x64
    2. build release win32 and x64
  7. Open BDB_dotNet_vs2010.sln (allow default conversion)
    1. build debug win32 and x64
    2. build release win32 and x64
  8. Open BDB_dotNet_examples_vs2010.sln
    1. In each project, delete the missing reference to "db_dotnet" and add reference to ..\AnyCPU\Release\libdb_dotnet52.dll
    2. build debug win32 and x64
    3. build release win32 and x64

That’s it. Now you can run the example projects and you have a .NET library and x86 (Win32) and x64 binary engines for that library to use. Enjoy!

Upgrade to System.Data.SQLite 1.0.74.0 in Visual Studio 2010

I wrote about getting SQLite up and working in Visual Studio 2010 and your .NET 4.0 projects in June. Then I got distracted with work and didn’t have a chance to come back to my exploration of SQLite until today. So I decided to look to see if we have a new version. Yes, the new System.Data.SQLite install for 1.0.74.0 (SQLite version 3.7.7.1 which includes a few bug fixes) is ready to download and install.

Unfortunately, the designers are still not available, so if you want any designer, however buggy, in Visual Studio, you’re still stuck with my original path in my previous post. (See the readme once you finish the 1.0.74.0 install.)

After a variety of experiments with the new installer, I ended up taking the following steps that led to successfully running my budding test/prototype application. Follow these steps and you won’t go too wrong:

  1. Uninstall all SQLite related items from Control Panel | Program and Features
  2. Download and install my favorite free SQLite admin tool from http://osenxpsuite.net/?xp=3.
  3. Download the latest from http://system.data.sqlite.org/index.html/doc/trunk/www/downloads.wiki. In my case, it was the sqlite-netFx40-setup-bundle-x64-2010-1.0.74.0.exe.
  4. This will have removed some GAC items and NOT put 1.0.74.0 into GAC. At least on my machine. I could not get the assemblies into the GAC to save my life.
  5. Make sure your project references are pointed to the correct version and that you have Copy Local set to true.
  6. Change your config settings to remove useLegacyV2RuntimeActivationPolicy=true from the startup node like this:
      <startup>
        <supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.0" />
      </startup>
  7. Change your config settings in DbProviderFactories to remove strong name reference like this:
      <system.data>
        <DbProviderFactories>
          <remove invariant="System.Data.SQLite" />
          <add name="SQLite Data Provider"
              invariant="System.Data.SQLite"
              description=".Net Framework Data Provider for SQLite"
              type="System.Data.SQLite.SQLiteFactory, System.Data.SQLite" />
        </DbProviderFactories>
      </system.data>
  8. Test to taste.

Office Outlook Add-In Development

I’ve long been interested in building add-ins for Microsoft Office but every time I’ve started to dive into it, I’ve been repelled by the challenges of understanding the Office COM interfaces and dealing with multiple versions of Office, especially for Outlook add-ins.

All of that may be about to change. I just ran into Add-In-Express.com, a tool vendor that for whatever reason I’ve not come across until now. From what I’ve seen on their web site, I’m very impressed.

With a single VS 2010 project template, you can now create an Office add-in with wrapper support for multiple Office products and multiple versions all from one unified library.

image

They do not offer a 30-day trial, so this is a blatant attempt to convince them to give me a reviewer’s copy to install and experiment on.

My primary attraction to this toolset is that I’m not a fan of writing tedious routine plumbing code to deal with multiple versions of a product upon which my software will need to be used. This is also the primary reason I’ve avoided delving into Office add-in development beyond reaching that initial level of prohibitive frustration.

I’m particularly interested in the following feature sets and should I be granted an reviewer’s license or if not and I can scrounge up the scratch for a license, I will be putting these to the test in a rewrite and expansion of a time tracking tool that I wrote as a stand alone WinForms app about nine years ago.

Features to check out list:

  • Version neutrality—many of the users for whom I write software are institutionally stuck on old versions of Microsoft Office. They have no choice but I still need to deliver a solution to them.
  • Context sensitive toolbars and ribbons for Outlook—my primary target for an add-in host, every time I’ve read about or talked with developers who deal with writing their own multi-version plumbing, I walk away from the notion of writing the Outlook add-in I’m thinking of. I really want to see if this tool can help me overcome my fears.
  • Visual designers—one of the most exciting aspects that I want to explore in this product is the visual designers for the ribbon UI, toolbars, options and folders properties dialogs, and especially the form and view regions for Outlook.
  • Smart Tags designer—I am keenly interested in natural language processing and enhancing the utility of email by analyzing text for keywords and other language constructs and then providing context aware actions directly from the Smart Tag to allow the user to take an action that might otherwise require multiple steps from multiple applications.

If you have had any experience with this toolset, I’d like to hear from you. As soon as I’m able to get my hands on it, I will begin posting my experiences with the tool as well as deployable examples from my evaluation and experimentation.

Target Framework and Platform Target: Get Your .NET Projects on the Same Page

It builds and runs fine on your local machine but when you deploy it to the server, all kinds of problems arise. Before you spent countless hours chasing ghosts, make sure you have your .NET projects on the same page, namely building for the right .NET Framework and targeting the right platform.

Open your project properties bag and try these first and then narrow it down. First, make sure your target frameworks for all your projects are in synch.

targetframework

Then make sure you give yourself as much latitude as you can by setting your platform target to Any CPU:

anycpu

Once you know your project and its dependencies are all on the same page and you still have problems on your deployment target, then go ahead and spend countless hours chasing ghosts.