SQL Server FileStream

　　以往我们对文件管理有两种方法：

1. 数据库只保存文件的路径，具体的文件保存在文件服务器(NFS)上，使用时，编程实现从文件服务器读取文件；
2. 将文件直接以varbinary(max)或image数据类型保存在数据库中。

　　上面两种文件存放方式都有问题：第一种方法因为会访问磁盘，故受I/O影响性能不是很好，而且不能很好的进行文件备份；第二种方法虽然解决了文件备份（数据库的备份）问题，但是由于字段的字节数太大，对数据库本身也会造成影响，性能也很低下。

　　微软在SQL Server 2008推出了一种新的方式 - FileStream，它不是一种新的数据类型，而是一种技术，它使SQL Server数据库引擎和NTFS文件系统成为了一个整体，它结合了上面两种方式的优点：FileStream使用NT系统来缓存文件数据，而对文件数据的操作可使用Transact-SQL语句对其进行插入、更新、查询、搜索和备份。

　　https://msdn.microsoft.com/en-us/library/gg471497(v=sql.110).aspx

　　FILESTREAM integrates the SQL Server Database Engine with an NTFS file system by storing varbinary(max) binary large object (BLOB) data as files on the file system. Transact-SQL statements can insert, update, query, search, and back up FILESTREAM data. Win32 file system interfaces provide streaming access to the data.

　　FILESTREAM uses the NT system cache for caching file data. This helps reduce any effect that FILESTREAM data might have on Database Engine performance. The SQL Server buffer pool is not used; therefore, this memory is available for query processing.

一、FileStream配置

1. 配置SQL Server安装实例：Start -> All Programs -> Microsoft SQL Server 2008 R2 -> Configuration Tools -> SQL Server Configuration Manager

　　右击属性，切换到FILESTREAM标签，勾选如下配置

　　2. 打开SQL Server，并配置如下

 　　以上也可以通过如下脚本执行：

Exec sp_configure filesteam_access_level, 2
RECONFIGURE

　　最后重启SQL Server Service

二、实例展示

　　创建FileStream类型文件/组

　　FILESTREAM data must be stored in FILESTREAM filegroups. A FILESTREAM filegroup is a special filegroup that contains file system directories instead of the files themselves. These file system directories are called data containers. Data containers are the interface between Database Engine storage and file system storage.

--Create filestreamgroup 
ALTER DATABASE [Archive]
ADD FILEGROUP [FileStreamGroup] CONTAINS FILESTREAM 
GO

--Create filestream and association with filestreamgroup above
ALTER DATABASE [Archive]
ADD FILE ( NAME = N'FileStream', FILENAME = N'D:CompanyDataSQL ServerFileStream') TO FILEGROUP [FileStreamGroup]
GO

　　filestream.hdr 文件是 FILESTREAM 容器的头文件。filestream.hdr 文件是重要的系统文件。它包含 FILESTREAM 标头信息。请勿删除或修改此文件。

　　创建测试表（注意：如果表包含FILESTREAM列，则每一行都必须具有唯一的行ID）

--Create table
CREATE TABLE Archive.dbo.Attachment (
    [ID] [UNIQUEIDENTIFIER] ROWGUIDCOL NOT NULL PRIMARY KEY,
    [FileName] NVARCHAR(100) NULL,
    [CreateUser] NVARCHAR(100) NULL,
    [CreateDatetime] DATETIME NULL,
    [Content] VARBINARY(MAX) FILESTREAM NULL  
)
FILESTREAM_ON [FileStreamGroup] 

　　插入一些测试数据

--Insert some records
INSERT INTO Attachment VALUES 
(NEWID(),'File Name 1','shg.cpan', GETDATE(),NULL),
(NEWID(),'File Name 1','shg.cpan', GETDATE(),CAST('' AS VARBINARY(MAX))),
(NEWID(),'File Name 1','shg.cpan', GETDATE(),CAST('This is a attachment, which contains all introduction for filestream' AS VARBINARY(MAX)))

　　从前台插入一些数据

using (SqlConnection conn = new SqlConnection("server=10.7.15.172;database=Archive;uid=sa;pwd=1234;Connect Timeout=180"))
{
    conn.Open();
    using (SqlCommand cmd = conn.CreateCommand())
    {
        string id = Guid.NewGuid().ToString();
        cmd.CommandText = "INSERT INTO Attachment VALUES('" + id + "','File Name 2','shg.cpan','" + DateTime.Now + "',@content)";
        SqlParameter param = new SqlParameter("@content", SqlDbType.VarBinary, 1000000000);
        param.Value = File.ReadAllBytes(@"D:Folder131 u_ex151207.log");
        cmd.Parameters.Add(param);
        cmd.ExecuteNonQuery();
    }
    conn.Close();
}

　　检索数据 

SELECT DATALENGTH(CONTENT)/(1024.0 * 1024.0) AS MB,* FROM ATTACHMENT

　　结果

　　文件系统

　　

　　上面的文件都是上传的真实文件，只不过没有后缀，如果重命名加上后缀，即可读取，如最后一个是excel文件，加上.xls，即可用Excel软件打开此文件。

　　下面我们再插入一条记录

INSERT INTO Attachment VALUES 
(NEWID(),'Win32API','shg.cpan', GETDATE(),CAST('This is a attachment, which contains all introduction for filestream' AS VARBINARY(MAX)))

　　

　　文件名00000016-0000016e-000c是如何和数据关联的呢

　　DBCC IND (Archive, Attachment, -1)

　　我们看一下PagePID为110的页情况（PageType为1表明是数据页）

DBCC TRACEON (3604);
DBCC PAGE (Archive, 1, 110, 3);
GO

　　看到了什么？CreateLSN即是我们在文件系统中看到的文件名00000016:0000016e:000c，这样数据库中的纪录就和文件联系起来了

三、使用 Win32 管理 FILESTREAM 数据

https://technet.microsoft.com/zh-cn/library/cc645940(v=sql.105).aspx

可以使用 Win32 在 FILESTREAM BLOB 中读取和写入数据。您需要执行以下步骤：

1. 读取 FILESTREAM 文件路径;
2. 读取当前事务上下文;
3. 获取 Win32 句柄，并使用该句柄在 FILESTREAM BLOB 中读取和写入数据

读取 FILESTREAM 文件路径

DECLARE @filePath varchar(max)

SELECT @filePath = Content.PathName()
FROM Archive.dbo.Attachment
WHERE FileName = 'Win32API'
PRINT @filepath

\CHUNTING-PCMSSQLSERVERv1ArchivedboAttachment\%!Content583FFDB4-921B-4340-8247-130174488DC8

读取当前事务上下文

DECLARE @txContext varbinary(max)

BEGIN TRANSACTION
SELECT @txContext = GET_FILESTREAM_TRANSACTION_CONTEXT()
PRINT @txContext
COMMIT

0x9D84E776FD943D419C99727C7AAA5B00

获取 Win32 句柄，并使用该句柄在 FILESTREAM BLOB 中读取和写入数据

若要获取 Win32 文件句柄，请调用 OpenSqlFilestream API。此 API 是从 sqlncli.dll 文件中导出的。可以将返回的句柄传递给以下任何 Win32 API：ReadFile、WriteFile、TransmitFile、SetFilePointer、SetEndOfFile 或 FlushFileBuffers。下面的示例说明了如何获取 Win32 文件句柄并使用它在 FILESTREAM BLOB 中读取和写入数据。

using System;
using System.Collections.Generic;
using System.Data;
using System.Data.SqlClient;
using System.Data.SqlTypes;
using System.IO;
using System.Linq;
using System.Security.AccessControl;
using System.Text;

namespace FileStreamConsoleApp
{
    class Program
    {
        static void Main(string[] args)
        {
            SqlConnection sqlConnection = new SqlConnection("server=10.7.15.172;database=Archive;Connect Timeout=180;Integrated Security=true");

            SqlCommand sqlCommand = new SqlCommand();
            sqlCommand.Connection = sqlConnection;

            try
            {
                sqlConnection.Open();

                //The first task is to retrieve the file path of the SQL FILESTREAM BLOB that we want to access in the application.
                sqlCommand.CommandText = "SELECT Content.PathName() FROM Archive.dbo.Attachment WHERE FileName = 'Win32API'";

                String filePath = null;

                Object pathObj = sqlCommand.ExecuteScalar();
                if (DBNull.Value != pathObj)
                {
                    filePath = (string)pathObj;
                }
                else
                {
                    throw new System.Exception("Chart.PathName() failed to read the path name for the Chart column.");
                }

                //The next task is to obtain a transaction context. All FILESTREAM BLOB operations occur within a transaction context to maintain data consistency.

                //All SQL FILESTREAM BLOB access must occur in a transaction. MARS-enabled connections have specific rules for batch scoped transactions,
                //which the Transact-SQL BEGIN TRANSACTION statement violates. To avoid this issue, client applications should use appropriate API facilities for transaction management, 
                //management, such as the SqlTransaction class.

                SqlTransaction transaction = sqlConnection.BeginTransaction("mainTranaction");
                sqlCommand.Transaction = transaction;

                sqlCommand.CommandText = "SELECT GET_FILESTREAM_TRANSACTION_CONTEXT()";

                Object obj = sqlCommand.ExecuteScalar();
                byte[] txContext = (byte[])obj;


                //The next step is to obtain a handle that can be passed to the Win32 FILE APIs.
                SqlFileStream sqlFileStream = new SqlFileStream(filePath, txContext, FileAccess.ReadWrite);

                byte[] buffer = new byte[512];

                int numBytes = 0;

                //Write the string, "EKG data." to the FILESTREAM BLOB. In your application this string would be replaced with the binary data that you want to write.

                string someData = "EKG data.";
                Encoding unicode = Encoding.GetEncoding(0);

                sqlFileStream.Write(unicode.GetBytes(someData.ToCharArray()), 0, someData.Length);

                //Read the data from the FILESTREAM BLOB.

                sqlFileStream.Seek(0L, SeekOrigin.Begin);

                numBytes = sqlFileStream.Read(buffer, 0, buffer.Length);

                string readData = unicode.GetString(buffer);

                if (numBytes != 0)
                {
                    Console.WriteLine(readData);
                }
                //Because reading and writing are finished, FILESTREAM must be closed. This closes the c# FileStream class, 
                //but does not necessarily close the the underlying FILESTREAM handle. 
                sqlFileStream.Close();

                //The final step is to commit or roll back the read and write operations that were performed on the FILESTREAM BLOB.

                sqlCommand.Transaction.Commit();
            }
            catch (System.Exception ex)
            {
                Console.WriteLine(ex.ToString());
            }
            finally
            {
                sqlConnection.Close();
            }
            Console.ReadKey();
        }
    }
}

注意此处的连接方式，必须是integrated security，如果设置成如下

SqlConnection conn = new SqlConnection("server=10.7.15.172;database=Archive;uid=sa;pwd=1234;Connect Timeout=180")

会报错提示权限问题

 

　　这是因为实际的文件保存在服务器的硬盘上，读取时读到的文件是"\xxxx"，还是通过网上邻居，所以要使用信任连接

Only the account under which the SQL Server service account runs is granted NTFS permissions to the FILESTREAM container. We recommend that no other account be granted permissions on the data container.

Note

SQL logins will not work with FILESTREAM containers. Only NTFS authentication will work with FILESTREAM containers.

另一篇文章 http://stackoverflow.com/questions/1398404/sql-server-filestream-access-denied

插入Attachment的完整C#语句

using System;
using System.Data;
using System.Data.SqlClient;
using System.Data.SqlTypes;
using System.IO;
using System.Transactions;

namespace FileStreamConsoleApp
{
    public class ArchiveAttachment
    {
        private const string ConnStr = "server=10.7.15.172;database=Archive;Connect Timeout=180;Integrated Security=true";

        public static void InsertAttachment(string Id, string fileName, string fileFullName, string createUser, DateTime createDatetime)
        {
            string InsertTSql = @"INSERT INTO Attachment(Id, FileName, CreateUser, CreateDatetime, Content) VALUES(@Id, @FileName, @CreateUser, @CreateDatetime, 0x); 
                                    SELECT Content.PathName(), GET_FILESTREAM_TRANSACTION_CONTEXT() FROM Attachment WHERE Id = @Id";

            string serverPath;
            byte[] serverTransaction;

            using (TransactionScope ts = new TransactionScope())
            {
                using (SqlConnection conn = new SqlConnection(ConnStr))
                {
                    conn.Open();

                    using (SqlCommand cmd = new SqlCommand(InsertTSql, conn))
                    {
                        cmd.Parameters.Add("@Id", SqlDbType.NVarChar).Value = Id;
                        cmd.Parameters.Add("@FileName", SqlDbType.NVarChar).Value = fileName;
                        cmd.Parameters.Add("@CreateUser", SqlDbType.NVarChar).Value = createUser;
                        cmd.Parameters.Add("@createDatetime", SqlDbType.DateTime).Value = createDatetime;
                        using (SqlDataReader sdr = cmd.ExecuteReader())
                        {
                            sdr.Read();
                            serverPath = sdr.GetSqlString(0).Value;
                            serverTransaction = sdr.GetSqlBinary(1).Value;
                            sdr.Close();
                        }
                    }
                    SaveAttachment(fileFullName, serverPath, serverTransaction);
                }
                ts.Complete();
            }
        }

        private static void SaveAttachment(string clientPath, string serverPath, byte[] serverTransaction)
        {
            const int BlockSize = 1024 * 512;

            using (FileStream source = new FileStream(clientPath, FileMode.Open, FileAccess.Read))
            {
                using (SqlFileStream dest = new SqlFileStream(serverPath, serverTransaction, FileAccess.Write))
                {
                    byte[] buffer = new byte[BlockSize];
                    int bytesRead;
                    while ((bytesRead = source.Read(buffer, 0, buffer.Length)) > 0)
                    {
                        dest.Write(buffer, 0, bytesRead);
                        dest.Flush();
                    }
                    dest.Close();
                }
                source.Close();
            }
        }
    }
}

四、使用场合

　　请注意以下事项：

　　并不是所有的文件存储都适合使用FileStream，如果所存储的文件对象平均大于1MB考虑使用FileStream，否则对于较小的文件对象，以varbinary(max)BLOB存储在数据库中通常会提供更为优异的流性能

https://msdn.microsoft.com/en-us/library/gg471497(v=sql.110).aspx

http://stackoverflow.com/questions/13420305/storing-files-in-sql-server#

http://research.microsoft.com/apps/pubs/default.aspx?id=64525

一些参考链接： 

使用FILESTREAM最佳实践：https://technet.microsoft.com/zh-cn/library/dd206979(v=sql.105).aspx

设计和实现 FILESTREAM 存储：https://technet.microsoft.com/zh-cn/library/bb895234%28v=sql.105%29.aspx?f=255&MSPPError=-2147217396

二进制大型对象 (Blob) 数据 (SQL Server)：https://technet.microsoft.com/zh-cn/library/bb895234(v=sql.110).aspx

Files and Filegroups Architecture：https://msdn.microsoft.com/en-us/library/ms179316.aspx?f=255&MSPPError=-2147217396

FILESTREAM Storage in SQL Server 2008：https://msdn.microsoft.com/en-us/library/hh461480.aspx?f=255&MSPPError=-2147217396