zoukankan      html  css  js  c++  java
  • [置顶] 如何在Azure存储空间(Storage)建立属于自己的备份

       大家好,我们知道Windows Azure平台上我们可以将自己的数据存在各式各样的结构中,例如Blob适合大文件和二进制流文件,Table适合存储一些较有结构层次的数据,Queue适合作为不同Role之间的通信消息存储,SQL Azure适合大型的关系数据库。这几种结构中SQL Azure作为一个独立的部分和其他3种有少许不同,SQL Azure本质上来收是一个分布式的SQL Server,除开Federation上的一些问题,其他部分和普通的SQL Server类似。而Blob,Table,Queue统称为Azure Storage,是一种云端的存储方式,Azure Storage中数据通常是比较安全的,Azure会将你的数据备份3份,一旦发生任何故障,Fabric control都会帮助你自动创建新的备份,所以不必担心因为断电或者硬件问题造成的数据丢失或错误。

      看到这里可能有些读者就比较疑惑了,之前你提到Azure会帮助我们做到自我备份,那么你还写这样一篇文章来说备份做什么呢? 其实不是这样的,Azure能帮你做到的是物理备份,用意是防止数据丢失,保证服务的稳定。对于你来说,你并不需要知道Azure是怎样为你的数据备份的,你能看到只是你自己数据而已。试想一下,就算你有3份备份在Azure storage上,如果此时你不小心误操作一些数据,Azure会按时间将你错误的数据统统复制3份去替代原先正确的数据,这样的话就前功尽弃了。

      这里我们提到的是逻辑意义上的备份,在你的数据非常重要且敏感的情况下,多做一份逻辑备份是非常有必要的,这里我们举一个例子来说明如何在Azure的平台上通过代码来做到逻辑备份,当然你也可以设置定时备份这样的功能。写代码之前还是那句话,需要安装一些Azure组件在你的机器里,这些是必不可少的准备工作:

    Azure 账号和 Storage账号(下文将会提到),没有的话可以去注册一个或者使用试用版,目前Azure看起来是已经登录中国了,大家可以关注一下。

    [本示例完整源码下载(0分)] http://download.csdn.net/detail/aa466564931/5483345

    首先我们创建一个Web Role,首先设计好你的UI,最最简单的是放置一个按钮就可以了,当然为了让我的页面看起来更佳美观和好用,我们还是放置更多一些的内容在页面上,首先说Blob的备份。

    HTML代码:

    <asp:Label ID="lbContent" runat="server" ForeColor="Red"></asp:Label>
            <br />
            <br />
            <div style="border: 2px solid;">
            <div style="font-style:italic;font-weight:bold">
            Back up Blob Storage 
            Name Rule:
            <p>
                - Container names must start with a letter or number, and can contain only letters, numbers, and the dash (-) character.
            </p>
            <p>
                - Every dash (-) character must be immediately preceded and followed by a letter or number; consecutive dashes are not permitted in container names.
            </p>
            <p>
                - All letters in a container name must be lowercase.
            </p>
            <p>
                Container names must be from 3 through 63 characters long.
            </p>
            <p> Check: 
                <a href="http://msdn.microsoft.com/en-us/library/windowsazure/dd135715.aspx">http://msdn.microsoft.com/en-us/library/windowsazure/dd135715.aspx</a>  </p>
            </div>
            <br />
            <br />
            Source Container Name:
            <asp:TextBox ID="tbSource" runat="server"></asp:TextBox>
            <br />
            Backup Container Name:
            <asp:TextBox ID="tbCopies" runat="server"></asp:TextBox>
             <br />
            <asp:Button ID="btnBackup" runat="server" Text="Back up your Blob" 
                onclick="btnBackup_Click" />
            <br />
             <br />
            <asp:Label ID="lbBackup" runat="server" ForeColor="Red"></asp:Label>
            </div>


     

    这里也提到当你创建Blob container时候必须遵守的一些命名规范(全为小写,长度3-63,禁止特殊字符除了“-”)。

    继续添加一个按钮:

        <asp:Button ID="btnUpload" runat="server" Text="Upload Resources to Storage" 
                onclick="btnUpload_Click" />


     

    接下来我们在Azure项目中写入我们的Azure账号信息,右键点击Azure项目-左侧配置(settings)- 添加配置(Add Settings)- 类型选择连接字符串(connection string)- 点击最右侧的按钮 弹出设置Storage对话框 Account name为你的账号 Account key为你的密码,账户和密码可以在你的Azure的账户中心找到,这里如果你需要更详细的信息,请参照:

    http://www.windowsazure.com/en-us/manage/services/storage/how-to-create-a-storage-account/

    http://www.windowsazure.com/en-us/develop/net/how-to-guides/blob-storage/#configure-access

    如果你是老用户,则可以跳过这一段。这里我们将这个connection string名字设为StorageConnections。(自定义)

    接着让我们通过代码来创建一个Azure account因为我们不需要每次连接storage的时候都去重新创建一边Account,那么我们将account设为一个全局变量:

            private CloudStorageAccount account;
            List<string> nameList = new List<string>() { "MSDN.jpg", "Microsoft.jpg" };
            protected void Page_Load(object sender, EventArgs e)
            {
                lbBackup.Text = string.Empty;
                lbContent.Text = string.Empty;
                account = CloudStorageAccount.FromConfigurationSetting("StorageConnections");
            }

    nameList只是我们需要备份的文件名,与创建account的过程无关。

    接着是初次上传文件的代码, 也就是需要被备份的原始数据 (2张图片):

            /// <summary>
            /// Upload resources to Storage.
            /// </summary>
            /// <param name="sender"></param>
            /// <param name="e"></param>
            protected void btnUpload_Click(object sender, EventArgs e)
            {
                try
                {
                    FileDataSource source = new FileDataSource("files");
                    CloudBlobClient client = account.CreateCloudBlobClient();
                    CloudBlobContainer container = client.GetContainerReference("blob");
                    container.CreateIfNotExist();
                    var permission = container.GetPermissions();
                    permission.PublicAccess = BlobContainerPublicAccessType.Container;
                    container.SetPermissions(permission);
                    bool flag = false;
    
                    foreach (string name in nameList)
                    {
                        if (!source.FileExists(name, "image"))
                        {
                            flag = true;
                            CloudBlob blob = container.GetBlobReference(name);
                            string path = string.Format("{0}/{1}", "Files", name);
                            blob.UploadFile(Server.MapPath(path));
    
                            FileEntity entity = new FileEntity("image");
                            entity.FileName = name;
                            entity.FileUrl = blob.Uri.ToString();
                            source.AddFile(entity);
                            lbContent.Text += String.Format("The image file {0} is uploaded successes. <br />", name);
                        }
                    }
                    if (!flag)
                        lbContent.Text = "You had uploaded these resources. The blob container name is 'blob', table name is 'files'";
                    else
                        lbContent.Text += "The blob container name is 'blob', The table name is 'files'";
                }
                catch (Exception ex)
                {
                    lbContent.Text = ex.Message;
                }
            }


    好了,接下来是备份按钮的代码,这里我创建了一个StorageManager类,用来检测Blob container和Blob是否存在,不存在则是无法备份了:

    StorageManger.cs

       public class StorageManager
        {
            /// <summary>
            /// Check CloudBlobContainer is exists.
            /// </summary>
            /// <param name="container"></param>
            /// <returns></returns>
            public static bool CheckIfExists(CloudBlobContainer container)
            {
                try
                {
                    container.FetchAttributes();
                    return true;
                }
                catch (StorageClientException e)
                {
                    if (e.ErrorCode == StorageErrorCode.ResourceNotFound)
                        return false;
                    else
                        throw;
                }
            }
    
            /// <summary>
            /// Check CloudBlob is exists.
            /// </summary>
            /// <param name="blob"></param>
            /// <returns></returns>
            public static bool CheckIfExists(CloudBlob blob)
            {
                try
                {
                    blob.FetchAttributes();
                    return true;
                }
                catch (StorageClientException e)
                {
                    if (e.ErrorCode == StorageErrorCode.ResourceNotFound)
                        return false;
                    else
                        throw;
                }
            }
        }


     

    backup按钮:

            protected void btnBackup_Click(object sender, EventArgs e)
            {
                try
                {
                    if (tbSource.Text.Trim().Equals(string.Empty) && tbCopies.Text.Trim().Equals(string.Empty))
                    {
                        lbBackup.Text = "Source TextBox and Copies TextBox can not be empty";
                        return;
                    }
    
                    string sourceContainerName = tbSource.Text.Trim();
                    string copiesContainerName = tbCopies.Text.Trim();
                    CloudBlobClient client = account.CreateCloudBlobClient();
    
                    CloudBlobContainer sourceContainer = client.GetContainerReference(sourceContainerName);
                    if (!StorageManager.CheckIfExists(sourceContainer))
                    {
                        lbBackup.Text = "The source blob container is not exists";
                        return;
                    }
                    CloudBlobContainer copiesContainer = client.GetContainerReference(copiesContainerName);
                    copiesContainer.CreateIfNotExist();
                    var permission = copiesContainer.GetPermissions();
                    permission.PublicAccess = BlobContainerPublicAccessType.Container;
                    copiesContainer.SetPermissions(permission);
    
    
                    foreach (var blob in sourceContainer.ListBlobs())
                    {
                        string uri = blob.Uri.AbsolutePath;
                        string[] matches = new string[] { "blob/" };
                        string FileName = uri.Split(matches, StringSplitOptions.None)[1].Substring(0);
                        CloudBlob sourceBlob = sourceContainer.GetBlobReference(FileName);
                        CloudBlob copiesBlob = copiesContainer.GetBlobReference(FileName);
                        copiesBlob.CopyFromBlob(sourceBlob);
                        lbBackup.Text += String.Format("The image file {0} is backup successes. Copies container name is {1} <br />", FileName, copiesContainerName);
                    }
                }
                catch (StorageClientException ex)
                {
                    if (ex.ExtendedErrorInformation.ErrorCode.Equals("OutOfRangeInput"))
                        lbBackup.Text = "Please check your blob container name.";
                    else
                        lbBackup.Text = ex.Message;
                }
                catch (Exception all)
                {
                    lbBackup.Text = all.Message;
                }
            }
    


    备份成功的blob会新创建好一个Blob,并且放在同一个BlobContainer下,你可以使用Storage Explorer检查一下是否备份成功了。

    刚才我们提到的是Blob的备份,blob你可以想象就是一个个单独的文件的备份,比较简单和直观,如果我们需要Table storage这种有一定结构性的数据该怎么办呢?也很简单,首先我们需要为Table Storage建立一个类库,这样调用和阅读起来都比较方便。接着建立好一个实体类,这里你可以想象成数据库里表结构对应实体类,不过唯一的不同是这个实体类需要继承TableServiceEntity类,并且给两个特殊key赋值,PartitionKey和RowKey,Rowkey你可以想象成是你的列的主键,这是不可以重复的,而partitionkey是作为你的Table的分区键使用的,对Table storage来说会有更好的性能。代码如下:

        public class FileEntity : TableServiceEntity
        {
            /// <summary>
            /// No parameters constructor
            /// </summary>
            public FileEntity()
            {
                PartitionKey = "all";
                RowKey = string.Format("{0:10}-{1}", DateTime.MaxValue.Ticks - DateTime.Now.Ticks, Guid.NewGuid()).Replace("-", "");
            }
    
            /// <summary>
            /// With parameters constructor
            /// </summary>
            /// <param name="partitionKey"></param>
            public FileEntity(string partitionKey)
            {
                PartitionKey = partitionKey;
                RowKey = string.Format("{0:10}-{1}", DateTime.MaxValue.Ticks - DateTime.Now.Ticks, Guid.NewGuid()).Replace("-", "");
            }
    
            public string FileName
            {
                get;
                set;
            }
    
            public string FileUrl
            {
                get;
                set;
            }
        }


     

    还需要一个用来创建查询的context类:

        public class FileContext : TableServiceContext
        {
            public FileContext(string baseAddress, StorageCredentials credentials)
                : base(baseAddress, credentials)
            {
    
            }
    
            /// <summary>
            /// Get all entities of table storage "files".
            /// </summary>
            public IEnumerable<FileEntity> GetEntities
            {
                get
                {
                    var list = this.CreateQuery<FileEntity>("files");
                    return list;
                }
            }
    
            /// <summary>
            /// Get all entities of table storage "files_backup".
            /// </summary>
            public IEnumerable<FileEntity> GetBackupEntities
            {
                get
                {
                    var list = this.CreateQuery<FileEntity>("files_backup");
                    return list;
                }
            }
        }


     

    最后我们可以建立Datasource类,方便view层和data层之间的相互调用(具体的方法都在这里,包括查询table,添加row,校验方法等等)

        public class FileDataSource
        {
            private static CloudStorageAccount account;
            private FileContext context;
            private string tableName;
    
            public FileDataSource(string tableName)
            {
                // Create table storage client via cloud account.
                account = CloudStorageAccount.FromConfigurationSetting("StorageConnections");
                CloudTableClient client = account.CreateCloudTableClient();
                client.CreateTableIfNotExist(tableName);
                this.tableName = tableName;
    
                // Table context properties.
                context = new FileContext(account.TableEndpoint.AbsoluteUri, account.Credentials);
                context.RetryPolicy = RetryPolicies.Retry(3, TimeSpan.FromSeconds(1));
                context.IgnoreResourceNotFoundException = true;
                context.IgnoreMissingProperties = true;
                
            }
    
            /// <summary>
            /// Get all entities method.
            /// </summary>
            /// <returns></returns>
            public IEnumerable<FileEntity> GetAllEntities()
            {
                var list = from m in this.context.GetEntities
                           select m;
                return list;
            }
    
            /// <summary>
            /// Get table rows by partitionKey.
            /// </summary>
            /// <param name="partitionKey"></param>
            /// <returns></returns>
            public IEnumerable<FileEntity> GetEntities(string partitionKey)
            {
                var list = from m in this.context.GetEntities
                           where m.PartitionKey == partitionKey
                           select m;
                return list;
            }
    
            /// <summary>
            /// Get specify entity.
            /// </summary>
            /// <param name="partitionKey"></param>
            /// <param name="fileName"></param>
            /// <returns></returns>
            public FileEntity GetEntitiesByName(string partitionKey, string fileName)
            {
                var list = from m in this.context.GetEntities
                           where m.PartitionKey == partitionKey && m.FileName == fileName
                           select m;
                if (list.Count() > 0)
                    return list.First<FileEntity>();
                else
                    return null;
            }
    
            /// <summary>
            /// Add an entity.
            /// </summary>
            /// <param name="entity"></param>
            public void AddFile(FileEntity entity)
            {
                this.context.AddObject(this.tableName, entity);
                this.context.SaveChanges();
            }
    
            /// <summary>
            /// Add multiple entities.
            /// </summary>
            /// <param name="entities"></param>
            public void AddNumbersOfFiles(List<FileEntity> entities)
            {
                if (entities.Count() > 0)
                {
                    int totleNumbers = entities.Count();
                    int uploadTimes = entities.Count() / 100;
                    if ((entities.Count() % 100) > 0)
                        uploadTimes += 1;
                    for (int i = 0; i < uploadTimes; i++)
                    {
                        if (i == uploadTimes - 1)
                        {
                            for (int j = i * 100; j < totleNumbers; j++)
                            {
                                this.context.AddObject(this.tableName, entities[j]);
                            }
                            this.context.SaveChanges();
                        }
                        else
                        {
                            for (int j = i * 100; j < (i + 1) * 100; j++)
                            {
                                this.context.AddObject(this.tableName, entities[j]);
                            }
                            this.context.SaveChanges();
                        }
                    }
                }
    
            }
    
            /// <summary>
            /// Make a judgment to check file is exists.
            /// </summary>
            /// <param name="filename"></param>
            /// <param name="partitionKey"></param>
            /// <returns></returns>
            public bool FileExists(string filename, string partitionKey)
            {
                IEnumerable<FileEntity> list = from m in this.context.GetEntities
                           where m.FileName == filename && m.PartitionKey == partitionKey
                           select m;
                if (list.Count()>0)
                {
                    return true;
                }
                else
                {
                    return false;
                }
            }
        }


     

    Table storage的按钮方法(同Blob,实现细节有少许不同,使用到了我们刚刚创建的TableStorageManager类库中的方法添加和查询Table Storage)

            protected void btnBackupTable_Click(object sender, EventArgs e)
            {
                try
                {
                    if (tbTabelSource.Text.Trim().Equals(string.Empty) && tbTableCopies.Text.Trim().Equals(string.Empty))
                    {
                        lbBackupTable.Text = "Source TextBox and Copies TextBox can not be empty";
                        return;
                    }
    
                    string sourceTableName = tbTabelSource.Text.Trim();
                    string copiesTableName = tbTableCopies.Text.Trim();
                    CloudTableClient client = account.CreateCloudTableClient();
    
    
                    if (!client.DoesTableExist(sourceTableName))
                    {
                        lbBackupTable.Text = "The source table is not exists";
                        return;
                    }
    
                    FileDataSource tableDataSource = new FileDataSource(sourceTableName);
                    List<FileEntity> sourceList = tableDataSource.GetAllEntities().ToList<FileEntity>();
                    client.DeleteTableIfExist(copiesTableName);
                    FileDataSource tableDataCopies = new FileDataSource(copiesTableName);
                    tableDataCopies.AddNumbersOfFiles(sourceList);
                    lbBackupTable.Text = String.Format("The source table {0} is backup successes. Copies table name is {1}", sourceTableName, copiesTableName);
                }
                catch (StorageClientException ex)
                {
                    if (ex.ExtendedErrorInformation.ErrorCode.Equals("OutOfRangeInput"))
                        lbBackupTable.Text = "Please check your blob container name.";
                    else
                        lbBackupTable.Text = ex.Message;
                }
                catch (Exception all)
                {
                    lbBackupTable.Text = all.Message;
                }
            }


    而Storage的另一个对象Queue来说通常是作为消息机制而存在的,所以我们一般不会对Queue做备份工作,如果你确实需要对Queue进行备份,参考Blob的代码,并且将Blob container改为Queue的相对读取和创建的方法即可。

    示例图片:

    1 点击上传资源按钮. (需要被备份的对象) blob container名字为blob, table名字为files


    2. 点击备份blob按钮 备份成功的blob container名字为blob-copy2

    3. 在Azure storage explorer工具中可以快速找到备份blob对象

    4. 点击table storage按钮 备份 新的table storage叫做filesCopy

    5.同样可以再Storage Explorer找到备份对象 你可以与源对象进行比较

  • 相关阅读:
    cs231n--详解卷积神经网络
    Spring 2017 Assignments2
    深度神经网络基础
    cs231n官方note笔记
    Spring 2017 Assignments1
    问题
    win7下解决vs2015新建项目,提示“未将对象引用设置到引用实例“的问题
    项目二:人脸识别
    ubutu强制关闭应用程序的方法
    将caj文件转化为pdf文件进行全文下载脚本(ubuntu下亲测有用)
  • 原文地址:https://www.cnblogs.com/javawebsoa/p/3109218.html
Copyright © 2011-2022 走看看