C# 玩转MongoDB（三）

前面两篇文章，已经讲解了C#对MongoDB的基本操作以及小文件的读写存储，那么对于大型（>=16M）文件呢？具体又该如何操作呢，本文主要以一个简单的小例子，简述C#如何通过GridFS进行MongoDB的大文件的操作，仅供学习分享使用，如有不足之处，还请指正。

什么是GridFS?

在实现GridFS方式前我先讲讲它的原理，为什么可以存大文件。驱动首先会在当前数据库创建两个集合："fs.files"和"fs.chunks"集合，前者记录了文件名，文件创建时间，文件类型等基本信息；后者分块存储了文件的二进制数据（并支持加密这些二进制数据）。分块的意思是把文件按照指定大小分割，然后存入多个文档中。"fs.files"怎么知道它对应的文件二进制数据在哪些块呢？那是因为在"fs.chunks"中有个"files_id"键，它对应"fs.files"的"_id"。"fs.chunks"还有一个键(int型)"n"，它表明这些块的先后顺序。这两个集合名中的"fs"也是可以通过参数自定义的。

GridFS存储原理

一个文件存储在两个集合中，一个用于存储元数据（文件名称，类型，大小等内容，可便于索引），一个用于存储真实二进制数据（分块存储），如下所示：

 GridFS安装

如果需要存储大型文件，则需要安装GridFS插件，如下所示：

项目--右键--管理Nuget程序包--打卡Nuget包管理器--浏览搜索MongoDB.Driver.GridFS--安装。如下所示：

示例截图

首先是文件的查询，如下所示：

文件的新增

 核心代码

本示例主要是在MongoDB中进行文件的操作，所以之前的MongoHelper已不再适用，本例新增了文件专用帮助类MongoFileHelper，如下所示：

using MongoDB.Bson;
using MongoDB.Driver;
using MongoDB.Driver.GridFS;
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Text;
using System.Threading.Tasks;

namespace DemoMongo.Common
{

    public class MongoFileHelper
    {

        private string connStr = "mongodb://127.0.0.1:27017";//服务器网址

        private string dbName = "hexdb";//数据库名称

        private IMongoClient client;//连接客户端

        private IMongoDatabase db;//连接数据库

        private string collName;//集合名称

        public MongoFileHelper()
        {

        }

        public MongoFileHelper(string connStr, string dbName, string collName)
        {
            this.connStr = connStr;
            this.dbName = dbName;
            this.collName = collName;
            this.Init();
        }

        /// <summary>
        /// 初始化连接客户端
        /// </summary>
        private void Init()
        {
            if (client == null)
            {
                client = new MongoClient(this.connStr);
            }
            if (db == null)
            {
                db = client.GetDatabase(this.dbName);
            }
        }

        /// <summary>
        /// 通过字节方式上传
        /// </summary>
        /// <param name="filePath"></param>
        public void UploadFile(string filePath)
        {
            IGridFSBucket bucket = new GridFSBucket(db);
            byte[] source = File.ReadAllBytes(filePath);
            string fileName = Path.GetFileName(filePath);
            var options = new GridFSUploadOptions
            {
                ChunkSizeBytes = 64512, // 63KB
                Metadata = new BsonDocument
                {
                    { "resolution", "1080P" },
                    { "copyrighted", true }
                }
            };
            var id = bucket.UploadFromBytes(fileName, source);
            //返回的ID，表示文件的唯一ID


        }

        /// <summary>
        /// 通过Stream方式上传
        /// </summary>
        /// <param name="filePath"></param>
        public void UploadFile2(string filePath)
        {
            IGridFSBucket bucket = new GridFSBucket(db);
            var stream = new FileStream(filePath, FileMode.Open);

            string fileName = Path.GetFileName(filePath);
            var options = new GridFSUploadOptions
            {
                ChunkSizeBytes = 64512, // 63KB
                Metadata = new BsonDocument
                {
                    { "resolution", "1080P" },
                    { "copyrighted", true }
                }
            };
            var id = bucket.UploadFromStream(fileName, stream);
            //返回的ID，表示文件的唯一ID


        }

        /// <summary>
        /// 通过字节写入到流
        /// </summary>
        /// <param name="filePath"></param>
        public void UploadFile3(string filePath)
        {
            IGridFSBucket bucket = new GridFSBucket(db);
            byte[] source = File.ReadAllBytes(filePath);
            string fileName = Path.GetFileName(filePath);
            var options = new GridFSUploadOptions
            {
                ChunkSizeBytes = 64512, // 63KB
                Metadata = new BsonDocument
                {
                    { "resolution", "1080P" },
                    { "copyrighted", true }
                }
            };
            using (var stream = bucket.OpenUploadStream(fileName, options))
            {
                var id = stream.Id;
                stream.Write(source, 0, source.Length);
                stream.Close();
            }
        }

        /// <summary>
        /// 下载文件
        /// </summary>
        /// <param name="id"></param>
        public void DownloadFile(ObjectId id,string filePath)
        {
            IGridFSBucket bucket = new GridFSBucket(db);
            byte[] source = bucket.DownloadAsBytes(id);
            //返回的字节内容
            //var bytes = await bucket.DownloadAsBytesAsync(id);
            using (Stream stream = new FileStream(filePath, FileMode.OpenOrCreate)) {
                stream.Write(source, 0, source.Length);
            }
        }

        public void DownloadFile2(ObjectId id)
        {
            IGridFSBucket bucket = new GridFSBucket(db);
            Stream destination = null;
            bucket.DownloadToStream(id, destination);
            //返回的字节内容
            //await bucket.DownloadToStreamAsync(id, destination);

        }

        public void DownloadFile3(ObjectId id)
        {
            IGridFSBucket bucket = new GridFSBucket(db);
            Stream destination = null;
            using (var stream = bucket.OpenDownloadStream(id))
            {
                // read from stream until end of file is reached
                stream.Close();
            }
        }

        public void DownloadFile4(string fileName)
        {
            IGridFSBucket bucket = new GridFSBucket(db);
            var bytes = bucket.DownloadAsBytesByName(fileName);

            // or

            Stream destination = null;
            bucket.DownloadToStreamByName(fileName, destination);

            // or

            using (var stream = bucket.OpenDownloadStreamByName(fileName))
            {
                // read from stream until end of file is reached
                stream.Close();
            }
        }

        public List<MongoFile> FindFiles()
        {
            IGridFSBucket bucket = new GridFSBucket(db);
            var filter = Builders<GridFSFileInfo>.Filter.And(
                //Builders<GridFSFileInfo>.Filter.Eq(x => x.Filename, string.Empty),
                Builders<GridFSFileInfo>.Filter.Gte(x => x.UploadDateTime, new DateTime(2015, 1, 1, 0, 0, 0, DateTimeKind.Utc)),
                Builders<GridFSFileInfo>.Filter.Lt(x => x.UploadDateTime, new DateTime(2022, 2, 1, 0, 0, 0, DateTimeKind.Utc)));
            var sort = Builders<GridFSFileInfo>.Sort.Descending(x => x.UploadDateTime);
            var options = new GridFSFindOptions
            {
                //Limit = 1,
                Sort = sort
            };
            List<MongoFile> lstFiles = new List<MongoFile>();
            using (var cursor = bucket.Find(filter, options))
            {
                var fileInfos = cursor.ToList();
                foreach (var fileInfo in fileInfos) {
                    MongoFile f = new MongoFile()
                    {
                         Id=fileInfo.Id,
                         name = fileInfo.Filename,
                         suffix = Path.GetExtension(fileInfo.Filename),
                        size = int.Parse(fileInfo.Length.ToString())
                    };
                    lstFiles.Add(f);
                }
            }
            return lstFiles;
        }

        public List<MongoFile> FindFileByName(string fileName)
        {
            IGridFSBucket bucket = new GridFSBucket(db);
            var filter = Builders<GridFSFileInfo>.Filter.And(
                Builders<GridFSFileInfo>.Filter.Eq(x => x.Filename, fileName),
                Builders<GridFSFileInfo>.Filter.Gte(x => x.UploadDateTime, new DateTime(2015, 1, 1, 0, 0, 0, DateTimeKind.Utc)),
                Builders<GridFSFileInfo>.Filter.Lt(x => x.UploadDateTime, new DateTime(2015, 2, 1, 0, 0, 0, DateTimeKind.Utc)));
            var sort = Builders<GridFSFileInfo>.Sort.Descending(x => x.UploadDateTime);
            var options = new GridFSFindOptions
            {
                //Limit = 1,
                Sort = sort
            };
            List<MongoFile> lstFiles = new List<MongoFile>();
            using (var cursor = bucket.Find(filter, options))
            {
                var fileInfos = cursor.ToList();
                foreach (var fileInfo in fileInfos)
                {
                    MongoFile f = new MongoFile()
                    {
                        Id = fileInfo.Id,
                        name = fileInfo.Filename,
                        suffix = Path.GetExtension(fileInfo.Filename),
                        size = int.Parse(fileInfo.Length.ToString())
                    };
                    lstFiles.Add(f);
                }
            }
            return lstFiles;
        }
    }
}

然后操作时，调用帮助类即可，如下所示：

查询调用

       private void btnQuery_Click(object sender, EventArgs e)
        {
            string name = this.txtName.Text.Trim();
            List<MongoFile> fileInfos = new List<MongoFile>();
            if (string.IsNullOrEmpty(name))
            {
                fileInfos = helper.FindFiles();
            }
            else {
                fileInfos = helper.FindFileByName(name);
            }
            
            this.dgView.AutoGenerateColumns = false;
            this.bsView.DataSource = fileInfos;
            this.dgView.DataSource = this.bsView;
        }

下载调用

        private void dgView_CellContentClick(object sender, DataGridViewCellEventArgs e)
        {
            if (e.ColumnIndex == 3) {
                //第3个是下载按钮
                SaveFileDialog sfd = new SaveFileDialog();
                
                var file = (MongoFile)(this.dgView.Rows[e.RowIndex].DataBoundItem);
                sfd.FileName = file.name;
                sfd.Title = "请保存文件";
                if (DialogResult.OK == sfd.ShowDialog())
                {
                    helper.DownloadFile(file.Id,sfd.FileName);
                    MessageBox.Show("保存成功");

                }
            }
        }

保存调用

        private void btnSave_Click(object sender, EventArgs e)
        {
            string filePath = this.txtPath.Text;
            if (!string.IsNullOrEmpty(filePath))
            {
                this.helper.UploadFile(filePath);
                MessageBox.Show("保存成功");
            }
            else {
                MessageBox.Show("请先选择文件");
            }
            
        }

MongoDB查询

当通过GridFS方式保存文件成功后，会在GridFS Buckets下生成fs对象，且在集合下生成两个集合【fs.files，fs.chunks】，用于存储文件，如下所示：

 通过查询fs.files集合，可以查找上传文件的列表，如下所示：

 通过查询fs.chunks集合，可以查询文件的内容(二进制数据)，如下所示：

 注意：如果文件太大，在fs.chunks集合中，进行分片存储，n表示存储的顺序。

 以上就是C#操作MongoDB大文件存储的相关内容，旨在抛砖引玉，共同进步。

备注

点绛唇·感兴

【朝代】宋代 【作者】王禹偁【chēng】

雨恨云愁，江南依旧称佳丽。水村渔市，一缕孤烟细。
天际征鸿，遥认行如缀。平生事，此时凝睇，谁会凭栏意。(栏 通：阑)