How to write a scalable TCP/IP based server

Asked 2009-05-15T16:49:18.670

152

I am in the design phase of writing a new Windows service application that accepts TCP/IP connections for long running connections (i.e., this is not like HTTP where there are many short connections, but rather a client connects and stays connected for hours or days or even weeks).

I'm looking for ideas for the best way to design the network architecture. I'm going to need to start at least one thread for the service. I am considering using the Asynch API (BeginRecieve, etc.) since I don't know how many clients I will have connected at any given time (possibly hundreds). I definitely do not want to start a thread for each connection.

Data will primarily flow out to the clients from my server, but there will be some commands sent from the clients on occasion. This is primarily a monitoring application in which my server sends status data periodically to the clients.

What is the best way to make this as scalable as possible? Basic workflow?

To be clear, I'm looking for .NET-based solutions (C# if possible, but any .NET language will work).

I would need a working example of a solution, either as a pointer to something I could download or a short example in-line. And it must be .NET and Windows based (any .NET language is acceptable).

Edit Report

2 Answers

Using .NET's integrated Async I/O (BeginRead, etc.) is a good idea if you can get all the details right. When you properly set up your socket/file handles it will use the OS's underlying IOCP implementation, allowing your operations to complete without using any threads (or, in the worst case, using a thread that I believe comes from the kernel's I/O thread pool instead of .NET's thread pool, which helps alleviate threadpool congestion.)

The main gotcha is to make sure that you open your sockets/files in non-blocking mode. Most of the default convenience functions (like File.OpenRead) don't do this, so you'll need to write your own.

One of the other main concerns is error handling - properly handling errors when writing asynchronous I/O code is much, much harder than doing it in synchronous code. It's also very easy to end up with race conditions and deadlocks even though you may not be using threads directly, so you need to be aware of this.

If possible, you should try and use a convenience library to ease the process of doing scalable asynchronous I/O.

Microsoft's Concurrency Coordination Runtime is one example of a .NET library designed to ease the difficulty of doing this kind of programming. It looks great, but as I haven't used it, I can't comment on how well it would scale.

For my personal projects that need to do asynchronous network or disk I/O, I use a set of .NET concurrency / I/O tools that I've built over the past year, called Squared.Task. It's inspired by libraries like imvu.task and twisted, and I've included some answered 2009-05-16T05:43:25.843

I would use the AcceptAsync/ConnectAsync/ReceiveAsync/SendAsync methods that were added in .NET 3.5. I have done a benchmark and they are approximately 35% faster (response time and bitrate) with 100 users constantly sending and receiving data.

answered 2009-05-21T23:38:03.930

How to write a scalable TCP/IP based server

2 Answers

Your Answer