网上对于java有很多开源的组件可以搭建分布式日志系统,我参考一些组件自己开发一套简单的分布式日志系
全部使用采用.NET进行开发,所用技术:MVC、EF、RabbitMq、MySql、Autofac
整体构架如下:
主要采用的RabbitMq进行处理,下面我主要讲讲.NET下运用RabbitMq
1.RabbitMQ Erlang 的安装,网上很多介绍,我这里就不多少了,请参考https://jingyan.baidu.com/article/a17d5285173ce68098c8f2e5.html
备注:安装好后一定注意保持.erlang.cookie这个文件的一致,在用户目录和windows下面。
2.RabbitMQ是使用,先安装RabbitMQ.Client,在VS中程序管理装入
RabbitMQ的辅助类
/// <summary> /// RabbitMQ消息队列处理 /// </summary> public class RabbitMQHelper { /// <summary> /// rabbitMQ地址 /// </summary> private string HostName = "localhost"; //ConfigurationManager.AppSettings["RabbitMQHostName"]; /// <summary> /// 账号 /// </summary> private string UserName = "guest"; //ConfigurationManager.AppSettings["RabbitMQUserName"]; /// <summary> /// 密码 /// </summary> private string Password = "guest"; // ConfigurationManager.AppSettings["RabbitMQPassword"]; /// <summary> /// 创建ConnectionFactory /// </summary> /// <returns></returns> private ConnectionFactory factory { get; set; } public RabbitMQHelper() { if (factory == null) { factory = new ConnectionFactory(); factory.HostName = HostName; factory.UserName = UserName; factory.Password = Password; } } public RabbitMQHelper(string UserName,string Password):base() { this.UserName = UserName; this.Password = Password; } public RabbitMQHelper(string UserName, string Password,string HostName) :base() { this.UserName = UserName; this.Password = Password; this.HostName = HostName; } /// <summary> /// 消息发送 /// </summary> /// <typeparam name="TEntity"></typeparam> /// <param name="entity"></param> /// <param name="MqName"></param> public void SendMsg<TEntity>(TEntity entity,string MqName) { if (entity == null || string.IsNullOrEmpty(MqName)) return; using (var connection = factory.CreateConnection()) { using (var channel = connection.CreateModel()) { bool durable = true; channel.QueueDeclare(MqName, durable, false, false, null); string message = Newtonsoft.Json.JsonConvert.SerializeObject(entity); //持久化队列消息 var properties = channel.CreateBasicProperties(); properties.Persistent = true; var body = Encoding.UTF8.GetBytes(message); channel.BasicPublish("", MqName, properties, body); } } } /// <summary> /// 接受消息 /// </summary> /// <typeparam name="TEntity"></typeparam> /// <param name="MqName"></param> /// <param name="entity"></param> /// <param name="action"></param> public void AcceptMsg<TEntity>(string MqName,out TEntity entity,Action action)where TEntity:class { entity =null; if (string.IsNullOrEmpty(MqName)) return; using (var connection = factory.CreateConnection()) { using (var channel = connection.CreateModel()) { bool durable = true; channel.QueueDeclare(MqName, durable, false, false, null); //公平分发 channel.BasicQos(0, 1, false); var consumer = new QueueingBasicConsumer(channel); channel.BasicConsume(MqName, false, consumer); while (true) { var ea = consumer.Queue.Dequeue(); var body = ea.Body; var message = Encoding.UTF8.GetString(body); entity = Newtonsoft.Json.JsonConvert.DeserializeObject<TEntity>(message); Thread.Sleep(1000); channel.BasicAck(ea.DeliveryTag, false); action(); } } } } }
3.在生成服务器只需要调用SendMsg将日志文件写入消息队列
4.在日志处理服务器采用 NLog进行日志持久化处理,目前采用2台服务,采用RabbitMQ公平分发到2台日志服务器中,最后进行持久化处理,NLog的使用这里就不作说明
如果对于日志处理有更好的方案欢迎指出,谢谢