tsJensen

A quest for software excellence...

Hopper<T>: Queue<T> Class in C# that Caches to Disk

In a recent project, I had a Windows Service that was receiving many thousands of requests which could only be processed at specific intervals. Soon I found that queuing so many requests in a simple Queue<T> class ran away with all my machine's memory.

So what to do?

I didn't really want to change my code. I like the Queue<T> class. So I wrote a class I call Hopper<T> which acts like Queue<T> but caches items to disk. It's constructor gives the class the information it needs to do all the work. Then you just use it like a Queue<T>.

public Hopper(
 
string cachePath,
 
string uniqueFileExtension,
 
int hopperCapacity,
 
int reorderLevel)

This class takes advantage of the System.Runtime.Serialization and the System.Runtime.Serialization.Formatters.Binary namespaces.

Download the code and give it a try. And be sure to let me know what you think.

Hopper.zip (2.74 KB)

Parsing HTML with C# by the Book (DTD)

Recently I had to write an HTML parser for a project I've been working on for some time now. First I tried translating an open source C++ parser but it really wasn't what I wanted and it was also under the GPL. After contacting the author and realizing (or re-remembering) that I could not use a GPL derivative in a commercial library or application, I scrapped that and went back to the source: the official HTML DTD.

Re-remembering how to read a DTD after not having done so for so long was a chore, but the folks at Autistic Cuckoo helped. So I found a very helpful tutorial. I spent the next day or two writing the code in the file you linked below. I took some inspiration from a few files I found while browsing the FireFox code under the Mozilla license. The rest of it came from studying the DTD and trying to figure out a way to encapsulate that in a usable object model.

Here's an example of how to use it:

HtmlDocument doc = new HtmlDocument(url, html);
StringBuilder sb = new StringBuilder();
Collection<HtmlTag> pcdata = doc.GetList(DtdElement.A);
foreach (HtmlTag tag in pcdata)
{
  if (!tag.EndTag)
  {
    Dictionary<string, string> attributes = doc.GetAttributes(tag);
    sb.AppendLine("");
    sb.AppendLine("A: " + doc.ReadSlice(tag.Slice));

    foreach (KeyValuePair<string, string> pair in attributes)
    {
      sb.AppendLine(" " + pair.Key + "=" + pair.Value);
    }
  }
}

I'm releasing it under the BSD license, which I like much more than the GPL as I'm not really a "true" free software zealot. The only think I ask is that if you fix a bug or make an improvement, please share it with me and I'll put up a new version here.  

NetBrick.Net.OpenUtils1.zip (35.31 KB)

Forget Fedora 5

Well, after struggling to get Fedora 5 to run on my machine and get the GUI up and running on an nVidia card, I've given up on this distribution after finding this bit of nasty news.

I think I'll try SUSE next. I've tried using the "YUM" updater and following a variety of instructions from a variety of posts to get my dual monitor eVGA GForce 7800 GT to work. All to no avail.

Once downloaded and installed, I'll post the results of my attempts with SUSE 10.

Venturing into Mono

I've begun the journey into Mono. Fedora 5 is nearly completely downloaded. I've freed up a partition on which to install it. I've downloaded the mono-1.1.1.13.6_0-installer.bin from the official site.

Why?

Because I'm building a system that must scale to many machines and we're considering using a virtual machine hosting system. And they only host virtual Linux boxes.

Will we definitely host the application on virtual system? No, not definitely. But if the port to Mono goes well, it's certainly an option.

My concerns about going to Mono is first, I know very little about Linux. Second, I'm using System.ServiceProcess.ServiceBase for my server, and that namespace, as far as I can tell, is not supported in Mono. So these two items may pose a bit of a learning curve.

After downloading some but not all of the Mono source files, I began wandering about and looking at how the Mono team has implemented various class libraries that we .NET developers take for granted every day. Talk about a wealth of code samples that will be extremely valuable in my daily work, regardless of whether I'm in Mono or MS .NET coding.

I'll post more on my progress into the world of Mono and Linux in the future. In the meantime, if you have any words of wisdom for me, please feel free...

SharpZipLib and Kudos to Scott Galloway

I had to get an object serialized to a byte[] and then compress it using SharpZipLib and gzip and then decompress it and deserialize it to the original object. The serialization was easy but for some reason I was struggling with using the GZipOutputStream and GZipInputStream compression providers in the SharpZipLib library.

Then I found Scott Galloway's compression helper. I highly recommend it. Anyway, here's the code without the helper. Visit Scott's blog for that.

 public class ResultSerializer
 {
  public static byte[] Serialize(ResultData data)
  {
   //convert to byte[]
   IFormatter frm = new BinaryFormatter();
   MemoryStream ms = new MemoryStream(8096);
   frm.Serialize(ms, data);
   byte[] serial = ms.ToArray();
   ms.Close();
   byte[] retval = ZipUtil.Compress(serial);
   return retval;
  }
  public static ResultData Deserialize(byte[] zipData)
  {
   byte[] data = ZipUtil.DeCompress(zipData);
   //now deserialize
   IFormatter frm = new BinaryFormatter();
   MemoryStream datams = new MemoryStream(data, 0, data.Length);
   ResultData retval = (ResultData)frm.Deserialize(datams);
   return retval;
  }
 }

Note that I renamed Scott's helper "ZipUtil" for my own reasons.

Seven Principles of Highly Effective Web 2.0

I very much enjoyed Dion's Thinking in Web 2.0 post. The ways to think in Web 2.0 seem to be growing with significant and useful comments. I would like to propose a side discussion that attempts to reduce Web 2.0 to seven specific principles.

The Highly Effective Web 2.0 is:

1. Specific - Purpose, content and interface is quickly understood.
2. Standard - Data is offered via open standards and protocols (i.e. HTML, XHTML, SOAP, RSS, SSL).
3. Transparent - Privacy and other policies are enforced and simple (see #1).
4. Accessible - Data should be easily found for those with and without disabilities.
5. Interactive - Participation is be encouraged and facilitated (see #1).
6. Inclusive - One thing leads to more like things rather than fewer.
7. Evolutionary - Everything is both familiar and new.

I tend to be overly verbose while clinging to the principle and value of brevity. If we are to understand the Web 2.0 wave, perhaps we can reduce it to seven (no more) principles that are stated simply and without the need for great expansion despite the fact that books may be written on the subject.

Please comment. Let me know which one(s) you would replace, with what, and why.

ASP.NET and IPC Remoting

I'm creating a .NET 2.0 ASP.NET web service as a front end to several Windows Services (also built using .NET 2.0) and want to use IPC since the web service and the Windows Services will be running on the same machine.

I don't want to use XML configuration files. I want to do it in code. It works with a console app to the Windows Service, but the ASP.NET web service blows chunks.

Failed to connect to an IPC port:  Access Denied

Search. Search. Search. One clue about "authorizedGroup" = "Everyone" but no code. Tinker. Stumble. Search. Tinker. Finally. Here's the final result in the Windows Service server:

Dictionary<string, string> props = new Dictionary<string, string>();
props.Add("authorizedGroup", "Everyone");
props.Add("portName", "ServerPortName");
serverChannel = new IpcServerChannel(props, null);
ChannelServices.RegisterChannel(serverChannel, true);
RemotingConfiguration.RegisterWellKnownServiceType(typeof(MarshalByRefObjectSubClass),
   "ServerAppName", WellKnownObjectMode.SingleCall);
serverChannel.StartListening(null);

With the client setup like this in the web service:

using System;
using System.Data;
using System.Configuration;
using System.Threading;
using System.Web;
using System.Web.Security;
using System.Web.UI;
using System.Web.UI.WebControls;
using System.Web.UI.WebControls.WebParts;
using System.Web.UI.HtmlControls;
using System.Runtime.Remoting;
using System.Runtime.Remoting.Channels;
using System.Runtime.Remoting.Channels.Ipc;
using MyRemotingInterfaces;

public class RemotingClientFactory
{
   private static Mutex mut = new Mutex();
   private static WellKnownClientTypeEntry remoteEntry;
   private static IpcClientChannel remoteChannel;
   private static string remoteUrl = "ipc://RemoteExampleRemoteServer/RemoteExampleRemote";

   static RemotingClientFactory() { }

   public static IMyRemoteObject CreateRemote()
   {
      if (remoteChannel == null || remoteEntry == null)
      {
         mut.WaitOne();
         try
         {
            if (remoteChannel == null)
            {
               remoteChannel = new IpcClientChannel();
               ChannelServices.RegisterChannel(remoteChannel, true);
            }
            if (remoteEntry == null)
            {
               remoteEntry =
                 new WellKnownClientTypeEntry(typeof(MyRemotingInterfaces.IMyRemoteObject),
                       remoteUrl);
               RemotingConfiguration.RegisterWellKnownClientType(remoteEntry);
            }
         }
         finally
         {
            mut.ReleaseMutex();
         }
      }
      try
      {
         IMyRemoteObject obj =
          
(IRemoteExampleRemote)Activator.GetObject(remoteEntry.ObjectType, remoteUrl);
         return obj;
      }
      catch(Exception e)
      {
         //TODO log then rethrow
         throw e;
      }
   }
}

And it works like a charm. It's not perfect, I'm sure. But it's a start. And it didn't seem like anyone had or wanted to post their solution to the newsgroups or anywhere else I could find.

Let me know if you find a better way or if this helps you. And good luck.

Easy Debug Windows Service

I know this problem has been solved many times and written about many times, but every time I go to create a new Windows Service project, I end up re-researching how to debug and step through code in a Windows Service project in Visual Studio.

I've done it now, again, building my first real set of Windows Service projects in Visual Studio 2005 and this time I took the compiler directive approach. It's been done before, sure, and many have written about it, but for my own short term memory's sake, here's my solution.

Step One
Create the Windows Service project using the New Project wizard and the Windows Service template.

Step Two
Modify the nicely created program.cs file as follows:

using System;
using System.Collections.Generic;
using System.ServiceProcess;
using System.Text;

namespace YourNameSpace
{
#if (DEBUG)
   class Program
#else
   static class Program
#endif
   {
      /// 
      /// The main entry point for the application.
      /// 
#if (DEBUG)
      static void Main(string[] args)
#else
      static void Main()
#endif
      {
#if (DEBUG)
         ServiceRunner sr = new ServiceRunner();
         sr.Start(args);
         Console.WriteLine("Started... Hit enter to stop...");
         Console.ReadLine();
         sr.Stop();
#else
         ServiceBase[] ServicesToRun;
         ServicesToRun = new ServiceBase[] { new Service1() };
         ServiceBase.Run(ServicesToRun);
#endif
      }
   }
}

Step Three
Modify the Service1.cs file as follows:

using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Diagnostics;
using System.ServiceProcess;
using System.Text;

namespace YourNameSpace
{
   public partial class Service1 : ServiceBase
   {
      private ServiceRunner serviceRunner = null;
      public Service1()
      {
         InitializeComponent();
         serviceRunner = new ServiceRunner();
      }

      protected override void OnStart(string[] args)
      {
         serviceRunner.Start(args);
      }

      protected override void OnStop()
      {
         serviceRunner.Stop();
      }
   }

   internal class ServiceRunner
   {
      public void Start(string[] args)
      {
         //TODO: Add code that will execute on start.
      }
      public void Stop()
      {
         //TODO: Add code that will execute on stop.
      }
   }
}

Step Four
Change the output type (in properties page of the project) to console application.

Now debug away!