tsJensen

A quest for software excellence...

BufferedStream Improves .NET Sockets Performance

.NET’s BufferedStream saved the day, instantly reducing 200-400ms operations across the wire to 1ms. Why? Simple answer, the Socket class returns a NetworkStream. Wire up that stream to a StreamWriter and a StreamReader, and you’re good to go, right? Wrong. Turns out the StreamWriter and StreamReader have a default 16 byte read/write buffer. And NetworkStream has none.

So if you have a TCP socket wired up to a NetworkStream, you’re trying to send or receive just 16 bytes at a time, utterly killing performance over TCP. Now magically wrap that NetworkStream into a BufferedStream and pass that BufferedStream into your StreamReader and StreamWriter and you get instant performance gains that will knock your sockets off.

Backstory: For months I’ve been writing and improving my DuoVia.Net fast services library. And recently the my day job’s team began using it some very clever ways (sorry, NDA and all), but we ran into a major performance problem. While performance on the same machine across processes using Named Pipes was excellent, the same was not true of machine-to-machine communications over TCP/IP. Sub-millisecond calls between services were taking 200-400ms across the wire. Something was terribly wrong. And when we tried Named Pipes from server to server, the performance problem went away. Of course, this was not the final answer because a .NET Named Pipes host can only handle 254 concurrent connections and we need to be able to scale beyond that.

Solving the problem required several sleepless nights and a weekend searching for the answer. My tests for TCP/IP with respect to performance have always run locally on the localhost loopback stack. The trouble with that, I have since learned (and should have known), is that when running locally, the Windows TCP stack bypasses the TCP stack altogether, or nearly so—sufficiently at least to mask the underlying problem of reading and writing only 16 bytes at a time directly on the NetworkStream.

After examining a number of open source implementations of Sockets on a server host, I ran into one or two smart enough to be using the BufferedStream to wrap that NetworkStream that a raw Socket object gives you. While doing all of this research, I also ran into the MSDN explanation (see Remarks and Example section) of how to improve server side asynchronous Socket handling. So I threw that into the solution as well. Once wired up and tested across machines on my home network, I breathed a huge sigh of relief. And here is what the code looks like now. First server and then client.

using System;
using System.Net.Sockets;
using System.Net;
using System.IO;
using System.Threading;
using System.Threading.Tasks;

namespace DuoVia.Net.TcpIp
{
  public class TcpHost : Host
  {
    private Socket _listener;
    private IPEndPoint _endPoint;
    private ManualResetEvent _listenResetEvent = new ManualResetEvent(false);

    /// <summary>
    /// Constructs an instance of the host and starts listening for 
		/// incoming connections on any ip address.
    /// All listener threads are regular background threads.
    /// </summary>
    /// <param name="port">The port number for incoming requests</param>
    /// <param name="log"></param>
    /// <param name="stats"></param>
    public TcpHost(int port, ILog log = null, IStats stats = null)
    {
      Initialize(new IPEndPoint(IPAddress.Any, port), log, stats);
    }

    /// <summary>
    /// Constructs an instance of the host and starts listening for incoming 
		/// connections on designated endpoint.
    /// All listener threads are regular background threads.
    /// 
    /// NOTE: the instance created from the specified type 
		/// is not automatically thread safe!
    /// </summary>
    /// <param name="endpoint"></param>
    /// <param name="log"></param>
    /// <param name="stats"></param>
    public TcpHost(IPEndPoint endpoint, ILog log = null, IStats stats = null)
    {
      Initialize(endpoint, log, stats);
    }

    private void Initialize(IPEndPoint endpoint, ILog log, IStats stats)
    {
      base.Log = log;
      base.Stats = stats;
      _endPoint = endpoint;
      _listener = new Socket(AddressFamily.InterNetwork, 
			  SocketType.Stream, ProtocolType.Tcp);
      _listener.SetSocketOption(SocketOptionLevel.Socket, 
			  SocketOptionName.KeepAlive, true);
      _listener.SetSocketOption(SocketOptionLevel.Socket, 
			  SocketOptionName.DontLinger, true);
    }

    /// <summary>
    /// Gets the end point this host is listening on
    /// </summary>
    public IPEndPoint EndPoint
    {
      get { return _endPoint; }
    }

    protected override void StartListener()
    {
      Task.Factory.StartNew(Listen, TaskCreationOptions.LongRunning);
    }

    private SocketAsyncEventArgs _acceptEventArg;

    /// <summary>
    /// Listens for incoming tcp requests.
    /// </summary>
    private void Listen()
    {
      try
      {
        _listener.Bind(_endPoint);
        _listener.Listen(8192);

        _acceptEventArg = new SocketAsyncEventArgs();
        _acceptEventArg.Completed 
				  += new EventHandler<SocketAsyncEventArgs>
					   (acceptEventArg_Completed);

        while (!_disposed)
        {
          // Set the event to nonsignaled state.
          _listenResetEvent.Reset();
          _acceptEventArg.AcceptSocket = null;
          try
          {
            if (!_listener.AcceptAsync(_acceptEventArg))
            {
              AcceptNewClient(_acceptEventArg);
            }
          }
          catch (Exception ex)
          {
            _log.Error("Listen error: {0}", 
						  ex.ToString().Flatten());
            break; //break loop on unhandled
          }

          // Wait until a connection is made before continuing.
          _listenResetEvent.WaitOne();
        }
      }
      catch (Exception e)
      {
        _log.Fatal("Listen fatal error: {0}", e.ToString().Flatten());
      }
    }

    private void acceptEventArg_Completed(object sender, 
		  SocketAsyncEventArgs e)
    {
      AcceptNewClient(e);
    }

    private void AcceptNewClient(SocketAsyncEventArgs e)
    {
      try
      {
        if (e.SocketError != SocketError.Success)
        {
          if (!_disposed) _listenResetEvent.Set();
          return;
        }

        Socket activeSocket = null;
        BufferedStream stream = null;
        try
        {
          activeSocket = e.AcceptSocket;

          // Signal the listening thread to continue.
          _listenResetEvent.Set();

          stream = new BufferedStream
					  (new NetworkStream(activeSocket), 8192);
          base.ProcessRequest(stream);
        }
        catch (Exception ex)
        {
          _log.Error("AcceptNewClient_ProcessRequest error: {0}", 
					  ex.ToString().Flatten());
        }
        finally
        {
          if (null != stream)
          {
            stream.Close();
          }
          if (null != activeSocket && activeSocket.Connected)
          {
            try
            {
              activeSocket.Shutdown(SocketShutdown.Both);
            }
            catch (Exception shutdownException)
            {
              _log.Error("AcceptNewClient_ActiveSocketShutdown error: {0}", 
							  shutdownException.ToString().Flatten());
            }

            try
            {
              activeSocket.Close();
            }
            catch (Exception closeException)
            {
              _log.Error("AcceptNewClient_ActiveSocketClose error: {0}", 
							  closeException.ToString().Flatten());
            }
          }
        }
      }
      catch (Exception fatalException)
      {
        _log.Fatal("AcceptNewClient fatal error: {0}", 
				  fatalException.ToString().Flatten());
      }
    }

    #region IDisposable Members

    private bool _disposed = false;

    protected override void Dispose(bool disposing)
    {
      if (!_disposed)
      {
        _disposed = true; //prevent second call to Dispose
        if (disposing)
        {
          _listenResetEvent.Set();
          _acceptEventArg.Dispose();
          _listener.Close();
          _listenResetEvent.Close();
        }
      }
      base.Dispose(disposing);
    }

    #endregion
  }
}

Client code:

using System;
using System.Net;
using System.Net.Sockets;
using System.Runtime.Serialization.Formatters.Binary;
using System.IO;

namespace DuoVia.Net.TcpIp
{
  public class TcpChannel : StreamingChannel
  {
    private Socket _client;

    /// <summary>
    /// Creates a connection to the concrete object handling 
    /// method calls on the server side
    /// </summary>
    /// <param name="serviceType"></param>
    /// <param name="endpoint"></param>
    public TcpChannel(Type serviceType, IPEndPoint endpoint)
    {
      _serviceType = serviceType;
      _client = new Socket(AddressFamily.InterNetwork, 
        SocketType.Stream, ProtocolType.Tcp);
      _client.LingerState.Enabled = false;
      _client.Connect(endpoint);
      if (!_client.Connected) throw new SocketException(); 
      _stream = new BufferedStream(new NetworkStream(_client), 8192);
      _binReader = new BinaryReader(_stream);
      _binWriter = new BinaryWriter(_stream);
      _formatter = new BinaryFormatter();
      SyncInterface(_serviceType);
    }

    public override bool IsConnected 
    { 
      get 
      { 
        return (null != _client) && _client.Connected; 
      } 
    }

    #region IDisposable override

    protected override void Dispose(bool disposing)
    {
      base.Dispose(disposing);
      if (disposing)
      {
        _binReader.Close();
        _binWriter.Close();
        _client.Close();
      }
    }

    #endregion
  }
}

You can find all of the ServiceWire code on GitHub or install the package from NuGet.