云计算之路-阿里云上：消灭“黑色n秒”第一招——不让CPU空闲团队

昨天对“黑色n秒”问题的最终猜想以失败而告终，从而让我们结束了被动猜想阶段，进入了主动进攻阶段——出招。

今天出第一招——用C#写个小程序，让其在每个CPU核上运行一个线程，不让任何一个CPU核进入空闲（idle）状态，以进一步排除CPU idle引起的“黑色n秒”。

在这一招中，借助的最重要的武器是System.Diagnostics.ProcessThread.ProcessorAffinity。通过给ProcessorAffinity设置一个掩码，就可以指定当前线程运行于哪个CPU核上。

如上图，用哪个核就把那个核对应的二进制位置1，其他位保持0。

所以对于我们所用的8核CPU，从第1核到第8核对应的ProcessorAffinity分别是：1, 2, 4, 8, 16, 32, 64, 128。

需要注意的地方是ProcessThread.ProcessorAffinity针对的是Windows操作系统线程，而.NET线程并不是与操作系统线程一一对应的，一个.NET线程可以运行于多个操作系统线程。所以，如果仅仅指定ProcessThread.ProcessorAffinity，并不能保证.NET线程运行于指定的CPU核上。那怎么办呢？

微软提供了解决方案，在设置ProcessThread.ProcessorAffinity之前需要通过下面的代码将.NET线程固定在操作系统线程上：

Thread.BeginThreadAffinity();

还有一个需要解决的问题是如何让一个线程一直处于执行状态，从而不让其所在的CPU核进入idle状态。微软也提供了解决方案，调用非托管方法SetThreadExecutionState()，代码如下：

NativeMethods.SetThreadExecutionState(NativeMethods.ES_CONTINUOUS | NativeMethods.ES_SYSTEM_REQUIRED);

下面请看招式：

代码第1部分：

class Program
{
    static void Main(string[] args)
    {
        var threads = new Thread[Environment.ProcessorCount];
        Console.WriteLine("Processor Count:" + Environment.ProcessorCount);
        for (int i = 0; i < threads.Length; i++)
        {
            var coreNumber = i + 1;
            var threaName = "ThreadOnCPU" + coreNumber;
            var start = new ThreadStart(() =>
            {
                Console.WriteLine(threaName + " is working...");
                NativeMethods.SetThreadExecutionState(
                    NativeMethods.ES_CONTINUOUS | NativeMethods.ES_SYSTEM_REQUIRED);
            });
            var thread = new DistributedThread(start);
            thread.ProcessorAffinity = (int)Math.Pow(2, i);
            thread.ManagedThread.Name = threaName;
            thread.Start();
        }
        Console.ReadKey();
    }       
}

internal static class NativeMethods
{
    [DllImport("kernel32.dll")]
    public static extern uint SetThreadExecutionState(uint esFlags);
    public const uint ES_CONTINUOUS = 0x80000000;
    public const uint ES_SYSTEM_REQUIRED = 0x00000001;
}

代码第2部分（来自Running .NET threads on selected processor cores ）：

class DistributedThread
{
    [DllImport("kernel32.dll")]
    public static extern int GetCurrentThreadId();

    [DllImport("kernel32.dll")]
    public static extern int GetCurrentProcessorNumber();

    private ThreadStart threadStart;

    private ParameterizedThreadStart parameterizedThreadStart;

    private Thread thread;

    public int ProcessorAffinity { get; set; }

    public Thread ManagedThread
    {
        get
        {
            return thread;
        }
    }

    private DistributedThread()
    {
        thread = new Thread(DistributedThreadStart);
    }

    public DistributedThread(ThreadStart threadStart)
        : this()
    {
        this.threadStart = threadStart;
    }

    public DistributedThread(ParameterizedThreadStart threadStart)
        : this()
    {
        this.parameterizedThreadStart = threadStart;
    }

    public void Start()
    {
        if (this.threadStart == null) throw new InvalidOperationException();

        thread.Start(null);
    }

    public void Start(object parameter)
    {
        if (this.parameterizedThreadStart == null) throw new InvalidOperationException();

        thread.Start(parameter);
    }

    private void DistributedThreadStart(object parameter)
    {
        try
        {
            // fix to OS thread
            Thread.BeginThreadAffinity();

            // set affinity
            if (ProcessorAffinity != 0)
            {
                CurrentThread.ProcessorAffinity = new IntPtr(ProcessorAffinity);
            }

            // call real thread
            if (this.threadStart != null)
            {
                this.threadStart();
            }
            else if (this.parameterizedThreadStart != null)
            {
                this.parameterizedThreadStart(parameter);
            }
            else
            {
                throw new InvalidOperationException();
            }
        }
        finally
        {
            // reset affinity
            CurrentThread.ProcessorAffinity = new IntPtr(0xFFFF);
            Thread.EndThreadAffinity();
        }
    }

    private ProcessThread CurrentThread
    {
        get
        {
            int id = GetCurrentThreadId();
            return
                (from ProcessThread th in Process.GetCurrentProcess().Threads
                    where th.Id == id
                    select th).Single();
        }
    }
}

接下来，出招——KeepAllCpuCoresAlive！

结果。。。这一招以失败告终！

（上图是“黑色1秒”发生时性能监视器监测到的ASP.NET Requests/Sec为0的情况）

失败又如何，就如同代码编译不通过一般不值一提。那为什么还要写博客出来呢？分享的就是过程！

接下来呢？准备第二招。。。

【参考资料】

Running .NET threads on selected processor cores

Threading in C#

Keep Alive the Machine - No Sleep