TThreadedServer vs. TNonblockingServer

Introduction

Which Thrift RPC server should MapKeeper use, TThreadedServer or TNonblockingServer? This benchmark compares 2 Thrift C++ RPC servers using StubServer. The focus of this benchmark is to test these 2 servers on a multi-core servers with a limited number (<1000) of concurrent client connections.

TThreadedServer

TThreadedServer spawns a new thread for each client connection, and each thread remains alive until the client connection is closed. This means that if there are 1000 concurrent client connections, TThreadedServer needs to run 1000 threads simultaneously.

TNonblockingServer

TNonblockingServer has one thread dedicated for network I/O. The same thread can also process requests, or you can create a separate pool of worker threads for request processing. The server can handle many concurrent connections with a small number of threads since it doesn’t need to spawn a new thread for each connection.

TThreadPoolServer (not benchmarked here)

TThreadPoolServer is similar to TThreadedServer; each client connection gets its own dedicated server thread. It’s different from TThreadedServer in 2 ways:

Server thread goes back to the thread pool after client closes the connection for reuse.
There is a limit on the number of threads. The thread pool won’t grow beyond the limit.

Client hangs if there is no more thread available in the thread pool. It’s much more difficult to use compared to the other 2 servers.

Configurations

Hardware

2 x Xeon E5620 2.40GHz (HT enabled, 8 cores, 16 threads)

Operating System

RHEL Server 5.4, Linux 2.6.18-164.2.1.el5 x86_64, 64-bit

Software

Thrift 0.6.1
TNonblockingServer thread pool size: 32 threads
Client and server run on the same box.

YCSB Workload

Number of client threads: 300
Number of requests: 10 million
Request size: ~60 bytes
Response size: ~30 bytes

Results

In this benchmark, TThreadedServer performs much better than TNonblockingServer. CPU is maxed out with TThreadedServer, while TNonblockingServer only uses about 20% of CPU time. I’m guessing it’s because the I/O thread is being the bottleneck and worker threads are not getting enough things to do .

Conclusion

TThreadedServer seems like a better fit for MapKeeper since I’m not planning to support thousands of concurrent connections (yet). TNonblockingServer might be a better choice when you face the C10K problem, but you need to make sure the I/O thread doesn’t become the bottleneck. It would be an interesting project to add a new type of Thrift server with a single accept() thread and multiple worker threads handling network I/O and request processing. There is already an open JIRA for this feature in Java. Is anybody interested in working on a similar feature in C++?

Comments?

Send me email at mapkeeper-users@googlegroups.com or post your comments here if you have any questions/suggestions.