C# 如何有效地异步登录？

How to effectively log asynchronously?

提问by Eric Schoonover

I am using Enterprise Library 4 on one of my projects for logging (and other purposes). I've noticed that there is some cost to the logging that I am doing that I can mitigate by doing the logging on a separate thread.

我在我的一个项目中使用 Enterprise Library 4 进行日志记录（和其他目的）。我注意到我正在做的日志记录有一些成本，我可以通过在单独的线程上进行日志记录来减轻这些成本。

The way I am doing this now is that I create a LogEntry object and then I call BeginInvoke on a delegate that calls Logger.Write.

我现在这样做的方法是创建一个 LogEntry 对象，然后在调用 Logger.Write 的委托上调用 BeginInvoke。

new Action<LogEntry>(Logger.Write).BeginInvoke(le, null, null);

What I'd really like to do is add the log message to a queue and then have a single thread pulling LogEntry instances off the queue and performing the log operation. The benefit of this would be that logging is not interfering with the executing operation and not every logging operation results in a job getting thrown on the thread pool.

我真正想做的是将日志消息添加到队列中，然后有一个线程将 LogEntry 实例从队列中拉出并执行日志操作。这样做的好处是日志不会干扰正在执行的操作，并且不是每个日志操作都会导致作业被抛出到线程池上。

How can I create a shared queue that supports many writers and one reader in a thread safe way? Some examples of a queue implementation that is designed to support many writers (without causing synchronization/blocking) and a single reader would be really appreciated.

如何以线程安全的方式创建一个支持多个作者和一个读者的共享队列？一些旨在支持多个写入器（不会导致同步/阻塞）和单个读取器的队列实现示例将非常受欢迎。

Recommendation regarding alternative approaches would also be appreciated, I am not interested in changing logging frameworks though.

关于替代方法的建议也将不胜感激，但我对更改日志框架不感兴趣。

采纳答案by Sam Saffron

I wrote this code a while back, feel free to use it.

我前段时间写了这段代码，请随意使用它。

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading;

namespace MediaBrowser.Library.Logging {
    public abstract class ThreadedLogger : LoggerBase {

        Queue<Action> queue = new Queue<Action>();
        AutoResetEvent hasNewItems = new AutoResetEvent(false);
        volatile bool waiting = false;

        public ThreadedLogger() : base() {
            Thread loggingThread = new Thread(new ThreadStart(ProcessQueue));
            loggingThread.IsBackground = true;
            loggingThread.Start();
        }


        void ProcessQueue() {
            while (true) {
                waiting = true;
                hasNewItems.WaitOne(10000,true);
                waiting = false;

                Queue<Action> queueCopy;
                lock (queue) {
                    queueCopy = new Queue<Action>(queue);
                    queue.Clear();
                }

                foreach (var log in queueCopy) {
                    log();
                }
            }
        }

        public override void LogMessage(LogRow row) {
            lock (queue) {
                queue.Enqueue(() => AsyncLogMessage(row));
            }
            hasNewItems.Set();
        }

        protected abstract void AsyncLogMessage(LogRow row);


        public override void Flush() {
            while (!waiting) {
                Thread.Sleep(1);
            }
        }
    }
}

Some advantages:

一些优点：

• It keeps the background logger alive, so it does not need to spin up and spin down threads.
• It uses a single thread to service the queue, which means there will never be a situation where 100 threads are servicing the queue.
• It copies the queues to ensure the queue is not blocked while the log operation is performed
• It uses an AutoResetEvent to ensure the bg thread is in a wait state
• It is, IMHO, very easy to follow

• 它使后台记录器保持活动状态，因此不需要启动和关闭线程。
• 它使用单个线程为队列提供服务，这意味着永远不会出现 100 个线程为队列提供服务的情况。
• 它复制队列以确保在执行日志操作时队列不被阻塞
• 它使用 AutoResetEvent 来确保 bg 线程处于等待状态
• 恕我直言，很容易理解

Here is a slightly improved version, keep in mind I performed very little testing on it, but it does address a few minor issues.

这是一个稍微改进的版本，请记住我对它进行了很少的测试，但它确实解决了一些小问题。

public abstract class ThreadedLogger : IDisposable {

    Queue<Action> queue = new Queue<Action>();
    ManualResetEvent hasNewItems = new ManualResetEvent(false);
    ManualResetEvent terminate = new ManualResetEvent(false);
    ManualResetEvent waiting = new ManualResetEvent(false);

    Thread loggingThread; 

    public ThreadedLogger() {
        loggingThread = new Thread(new ThreadStart(ProcessQueue));
        loggingThread.IsBackground = true;
        // this is performed from a bg thread, to ensure the queue is serviced from a single thread
        loggingThread.Start();
    }


    void ProcessQueue() {
        while (true) {
            waiting.Set();
            int i = ManualResetEvent.WaitAny(new WaitHandle[] { hasNewItems, terminate });
            // terminate was signaled 
            if (i == 1) return; 
            hasNewItems.Reset();
            waiting.Reset();

            Queue<Action> queueCopy;
            lock (queue) {
                queueCopy = new Queue<Action>(queue);
                queue.Clear();
            }

            foreach (var log in queueCopy) {
                log();
            }    
        }
    }

    public void LogMessage(LogRow row) {
        lock (queue) {
            queue.Enqueue(() => AsyncLogMessage(row));
        }
        hasNewItems.Set();
    }

    protected abstract void AsyncLogMessage(LogRow row);


    public void Flush() {
        waiting.WaitOne();
    }


    public void Dispose() {
        terminate.Set();
        loggingThread.Join();
    }
}

Advantages over the original:

与原版相比的优势：

• It's disposable, so you can get rid of the async logger
• The flush semantics are improved
• It will respond slightly better to a burst followed by silence

• 它是一次性的，所以你可以摆脱异步记录器
• 刷新语义得到改进
• 它会对一阵阵的反应稍好一点，然后是沉默

回答by Steve Gilham

An extra level of indirection may help here.

额外的间接级别可能会有所帮助。

Your first async method call can put messages onto a synchonized Queue and set an event -- so the locks are happening in the thread-pool, not on your worker threads -- and then have yet another thread pulling messages off the queue when the event is raised.

您的第一个异步方法调用可以将消息放入同步队列并设置一个事件——因此锁定发生在线程池中，而不是在您的工作线程上——然后有另一个线程在事件发生时将消息从队列中拉出被提出。

回答by leppie

If you log something on a separate thread, the message may not be written if the application crashes, which makes it rather useless.

如果你在一个单独的线程上记录一些东西，如果应用程序崩溃，消息可能不会被写入，这使得它变得毫无用处。

The reason goes why you should always flush after every written entry.

原因在于为什么您应该在每次写入后始终刷新。

回答by Jon Skeet

Yes, you need a producer/consumer queue. I have one example of this in my threading tutorial - if you look my "deadlocks / monitor methods"page you'll find the code in the second half.

是的，您需要一个生产者/消费者队列。我在我的线程教程中有一个这样的例子——如果你查看我的“死锁/监控方法”页面，你会在后半部分找到代码。

There are plenty of other examples online, of course - and .NET 4.0 will ship with one in the framework too (rather more fully featured than mine!). In .NET 4.0 you'd probably wrap a ConcurrentQueue<T>in a BlockingCollection<T>.

当然，网上还有很多其他示例——而且 .NET 4.0 也将在框架中附带一个（功能比我的更全面！）。在 .NET 4.0 中，您可能会将 a 包装ConcurrentQueue<T>在BlockingCollection<T>.

The version on that page is non-generic (it was written a longtime ago) but you'd probably want to make it generic - it would be trivial to do.

该页面上的版本是非通用的（它是很久以前编写的），但您可能希望使其通用 - 这样做很简单。

You would call Producefrom each "normal" thread, and Consumefrom one thread, just looping round and logging whatever it consumes. It's probably easiest just to make the consumer thread a background thread, so you don't need to worry about "stopping" the queue when your app exits. That does mean there's a remote possibility of missing the final log entry though (if it's half way through writing it when the app exits) - or even more if you're producing faster than it can consume/log.

您可以Produce从每个“正常”线程和Consume一个线程调用，只是循环并记录它消耗的任何内容。让消费者线程成为后台线程可能是最简单的，因此您无需担心在应用程序退出时“停止”队列。这确实意味着很可能会丢失最终的日志条目（如果应用程序退出时它已经写了一半） - 或者如果你的生产速度比它可以消耗/记录的速度更快。

回答by Corey Trager

If what you have in mind is a SHARED queue, then I think you are going to have to synchronize the writes to it, the pushes and the pops.

如果你想到的是一个共享队列，那么我认为你将不得不同步写入它、推送和弹出。

But, I still think it's worth aiming at the shared queue design. In comparison to the IO of logging and probably in comparison to the other work your app is doing, the brief amount of blocking for the pushes and the pops will probably not be significant.

但是，我仍然认为值得关注共享队列设计。与日志记录的 IO 相比，并且可能与您的应用程序正在执行的其他工作相比，推送和弹出的短暂阻塞可能并不重要。

回答by Rinat Abdullin

I suggest to start with measuring actual performance impact of loggingon the overall system (i.e. by running profiler) and optionally switching to something faster like log4net(I've personally migrated to it from EntLib logging a long time ago).

我建议首先测量日志记录对整个系统的实际性能影响（即通过运行分析器），并可选择切换到更快的东西，如log4net（我很久以前亲自从 EntLib 日志记录迁移到它）。

If this does not work, you can try using this simple method from .NET Framework:

如果这不起作用，您可以尝试使用 .NET Framework 中的这个简单方法：

ThreadPool.QueueUserWorkItem

Queues a method for execution. The method executes when a thread pool thread becomes available.

排队执行的方法。该方法在线程池线程可用时执行。

MSDN Details

MSDN 详细信息

If this does not work either then you can resort to something like John Skeet has offered and actually code the async logging framework yourself.

如果这也不起作用，那么您可以求助于 John Skeet 提供的类似的东西，并实际自己编写异步日志框架。

回答by Eric Schoonover

Here is what I came up with... also see Sam Saffron's answer. This answer is community wiki in case there are any problems that people see in the code and want to update.

这是我想出的……另请参阅 Sam Saffron 的回答。这个答案是社区维基，以防人们在代码中看到任何问题并想要更新。

/// <summary>
/// A singleton queue that manages writing log entries to the different logging sources (Enterprise Library Logging) off the executing thread.
/// This queue ensures that log entries are written in the order that they were executed and that logging is only utilizing one thread (backgroundworker) at any given time.
/// </summary>
public class AsyncLoggerQueue
{
    //create singleton instance of logger queue
    public static AsyncLoggerQueue Current = new AsyncLoggerQueue();

    private static readonly object logEntryQueueLock = new object();

    private Queue<LogEntry> _LogEntryQueue = new Queue<LogEntry>();
    private BackgroundWorker _Logger = new BackgroundWorker();

    private AsyncLoggerQueue()
    {
        //configure background worker
        _Logger.WorkerSupportsCancellation = false;
        _Logger.DoWork += new DoWorkEventHandler(_Logger_DoWork);
    }

    public void Enqueue(LogEntry le)
    {
        //lock during write
        lock (logEntryQueueLock)
        {
            _LogEntryQueue.Enqueue(le);

            //while locked check to see if the BW is running, if not start it
            if (!_Logger.IsBusy)
                _Logger.RunWorkerAsync();
        }
    }

    private void _Logger_DoWork(object sender, DoWorkEventArgs e)
    {
        while (true)
        {
            LogEntry le = null;

            bool skipEmptyCheck = false;
            lock (logEntryQueueLock)
            {
                if (_LogEntryQueue.Count <= 0) //if queue is empty than BW is done
                    return;
                else if (_LogEntryQueue.Count > 1) //if greater than 1 we can skip checking to see if anything has been enqueued during the logging operation
                    skipEmptyCheck = true;

                //dequeue the LogEntry that will be written to the log
                le = _LogEntryQueue.Dequeue();
            }

            //pass LogEntry to Enterprise Library
            Logger.Write(le);

            if (skipEmptyCheck) //if LogEntryQueue.Count was > 1 before we wrote the last LogEntry we know to continue without double checking
            {
                lock (logEntryQueueLock)
                {
                    if (_LogEntryQueue.Count <= 0) //if queue is still empty than BW is done
                        return;
                }
            }
        }
    }
}

回答by Johnny Serrano

In response to Sam Safrons post, I wanted to call flush and make sure everything was really finished writting. In my case, I am writing to a database in the queue thread and all my log events were getting queued up but sometimes the application stopped before everything was finished writing which is not acceptable in my situation. I changed several chunks of your code but the main thing I wanted to share was the flush:

为了回应 Sam Safrons 的帖子，我想打电话给flush 并确保一切都真的写完了。就我而言，我正在写入队列线程中的数据库，并且我的所有日​​志事件都在排队，但有时应用程序在所有写入完成之前就停止了，这在我的情况下是不可接受的。我更改了您的几段代码，但我想分享的主要内容是刷新：

public static void FlushLogs()
        {   
            bool queueHasValues = true;
            while (queueHasValues)
            {
                //wait for the current iteration to complete
                m_waitingThreadEvent.WaitOne();

                lock (m_loggerQueueSync)
                {
                    queueHasValues = m_loggerQueue.Count > 0;
                }
            }

            //force MEL to flush all its listeners
            foreach (MEL.LogSource logSource in MEL.Logger.Writer.TraceSources.Values)
            {                
                foreach (TraceListener listener in logSource.Listeners)
                {
                    listener.Flush();
                }
            }
        }

I hope that saves someone some frustration. It is especially apparent in parallel processes logging lots of data.

我希望这能避免一些人的挫败感。这在记录大量数据的并行进程中尤为明显。

Thanks for sharing your solution, it set me into a good direction!

感谢您分享您的解决方案，它让我找到了一个好的方向！

--Johnny S

——约翰尼·S

回答by Johnny Serrano

I wanted to say that my previous post was kind of useless. You can simply set AutoFlush to true and you will not have to loop through all the listeners. However, I still had crazy problem with parallel threads trying to flush the logger. I had to create another boolean that was set to true during the copying of the queue and executing the LogEntry writes and then in the flush routine I had to check that boolean to make sure something was not already in the queue and the nothing was getting processed before returning.

我想说我之前的帖子有点没用。您可以简单地将 AutoFlush 设置为 true，而不必遍历所有侦听器。但是，我仍然在尝试刷新记录器的并行线程方面遇到了疯狂的问题。我必须创建另一个布尔值，该布尔值在复制队列和执行 LogEntry 写入期间设置为 true，然后在刷新例程中我必须检查该布尔值以确保队列中没有某些内容并且没有处理任何内容在回来之前。

Now multiple threads in parallel can hit this thing and when I call flush I know it is really flushed.

现在并行的多个线程可以击中这个东西，当我调用 flush 我知道它真的被刷新了。

     public static void FlushLogs()
    {
        int queueCount;
        bool isProcessingLogs;
        while (true)
        {
            //wait for the current iteration to complete
            m_waitingThreadEvent.WaitOne();

            //check to see if we are currently processing logs
            lock (m_isProcessingLogsSync)
            {
                isProcessingLogs = m_isProcessingLogs;
            }

            //check to see if more events were added while the logger was processing the last batch
            lock (m_loggerQueueSync)
            {
                queueCount = m_loggerQueue.Count;
            }                

            if (queueCount == 0 && !isProcessingLogs)
                break;

            //since something is in the queue, reset the signal so we will not keep looping

            Thread.Sleep(400);
        }
    }

回答by John

Just an update:

只是一个更新：

Using enteprise library 5.0 with .NET 4.0 it can easily be done by:

将企业库 5.0 与 .NET 4.0 一起使用，可以通过以下方式轻松完成：

static public void LogMessageAsync(LogEntry logEntry)
{
    Task.Factory.StartNew(() => LogMessage(logEntry)); 
}

See: http://randypaulo.wordpress.com/2011/07/28/c-enterprise-library-asynchronous-logging/

见：http: //randypaulo.wordpress.com/2011/07/28/c-enterprise-library-asynchronous-logging/