什么是SharePoint 2013中的Shredded Storage?

Shredded storage(碎片存储)是SharePoint 2013中与大二进制对象(比方说微软PPT, Word文档一类的BLOB)管理相关的新提升.

Shredded Storage可以在SharePoint 2013中的文档增量修改或存储的时候既节省IO, 又降低计算资源的利用. Shredded Storage是建立在Cobalt(比如基于SOAP或HTTP的文件同步-File Synchronization via SOAP of HTTP)协议上的, 该协议是SharePoint 2010中引入的.

Cobalt

=======================

在SharePoint 2010中, 当你保存一个文档的时候, 比如说用Office客户端打开SharePoint中的文档, 仅仅是文档增量修改的部分会在网络上传输并由客户端提交给服务器. 然而, 文档在Web Server上会被合并, 并且需要从数据库完成一次完整的读操作, 然后包括了修改的新文件会被写入到数据库服务器中.

Shredded Storage 最基本的设计是确保更新文档的写操作的代价与修改的大小成比例, 而不是与文件的大小成比例.

SharePoint 2013允许内容被存储在巨大的流中, 也可以存储在互不相关的一堆Blob(shredded storage)中. 比方说我们有一个叫做Document.docx的文档, 与此文档关联的存储碎片会分布在一系列的BLOBs. 互不相关的BLOBs每个都被赋予一个独一无二的ID(offset)让文档被请求的时候可以用正确的顺序重新构造文档.

在SharePoint 2010中, 当一个文件被上传到文档库或列表中之后, 会在AllDocStreams表中创建单独的一行来存放所上传文件的BLOB信息. 正如前面所讨论的, 在后续对文档的更新中, 仅仅是更新了的字节(增量更新)会通过网络发送到服务器上, 这样的目的是减少网络资源的使用, 然而, 为了合并这些更新, 整个文档都会从数据库服务器中读取出来, 放到web server上, 在web server上将更新合并(merge)后发送会数据库服务器存储起来. 在SharePoint 2010中, 这个过程增强了文档的可靠性, 但是web server确是受害者. 在SharePoint 2010基础上的Shredded Storege模型有了提高, 原理是将单个的blog打碎为多个"shredded blobs", 存储在数据库的一张新表DocStream中. 每个Blog都包含一个数字id, 标识合并时后需要的Blob来源. 当客户端更新一个文档的时候, 仅仅是与修改相关的shredded blob 会更新到数据库, 而不是web server. 结果呢, 跟SharePoint 2010比FSSHTTP翻了倍, 而且存储空间也显著的减少了.

关于原理和内部实现等更多细节, 请查看原文.

一些要点:

Q: Can I disable Shredded Storage?

A: No, Shredded Storage is enabled by default and cannot be disabled.

Q: Can I prevent a file from being shredded?

A: Yes. In the event shredding is not desired the FileWriteChunkSize property can be set to the MaxFileSize of 2GB resulting in a monolithic file.

The following PowerShell commands will disable prevent a file from being shredded in scope of web application.

The default value of FileWriteChunkSize property is 64320, which is 64 KB.

 
$wa = Get-SPWebApplication http://portal/
$wa.WebService.FileWriteChunkSize 
$wa.WebService.FileWriteChunkSize = 1073741824
$wa.webservice.update() 

Q: If I use the PowerShell script above and changed the FileWriteChunkSize setting, will the shredded filed be combined back to one monolithic stream?

A: No, what have been shredded, stay shredded. This answer has been proved by our local test.

The following SQL Query could be used to verify whether a file has been shredded.

select * 
from DocsToStreams
where docid = (
select top 1 id 
from AllDocs 
where leafname like '%ThisIsFileName.doc%')

select * from DocsToStreams
where docid = (
select top 1 id 
from AllDocs 
where leafname like '%ThisisFileName.doc%')

Q: Is it true that bigger the file is, more blobs there will be?

A: Yes, it is. We have a 65 KB file, and it is shredded into 8 Blobs.  We have another 3.5 MB file, and it is shredded into 29 Blobs.

参考资料

=======================

Introduction to Shredded Storage in SharePoint 2013

http://blogs.technet.com/b/wbaer/archive/2012/11/12/introduction-to-shredded-storage-in-sharepoint-2013.aspx

Shredded Storage and the Evolution of SharePoint’s Storage Architecture

http://blogs.technet.com/b/wbaer/archive/2012/12/20/shredded-storage-and-the-evolution-of-sharepoint-s-storage-architecture.aspx