zoukankan      html  css  js  c++  java
  • 用 C# 实现 HTTP 协议多线程下载文件

    本文内容

    • 环境
    • Internet 请求
    • 演示
    • 参考资料
    • 修改记录

    环境


    • 开发工具:VS 2010/.NET Framework 4.0
    • 系统环境:Microsoft Windows 7

    Internet 请求


    应用程序通过 WebRequest.Create 方法创建 WebRequest 实例。该方法是静态方法,基于传递的 URI 创建从 WebRequest 派生的类。

    NET Framework 提供 HttpWebRequest 类,它派生自 WebRequest,来处理 HTTP 和 HTTPS 请求。在大多数情况下,WebRequest 类提供了你发出请求的所有属性,但是,如果需要的话,你可以把 WebRequest 对象强制类型转换成 HttpWebRequest,以访问请求的 HTTP 属性。类似地,HttpWebResponse 对象来处理 HTTP 和 HTTPS 请求的响应。若访问 HttpWebResponse 对象的属性,需要把 WebResponse 对象强制类型转换成 HttpWebResponse。

    .NET Framework 还提供了 FileWebRequestFileWebResponse 类,来处理使用 "file:" 资源的请求。类似地,FtpWebRequestFtpWebResponse 类用户 "ftp:"。

    若处理使用应用程序协议的请求,需要实现从 WebRequest 和 WebResponse 派生的特定协议类。

    说明:不能混淆 HttpWebResponse 和 HttpResponse 类;后者用于 ASP.NET 应用程序,而且它的方法和属性是通过 ASP.NET 的内部 Response 对象公开的。

    演示


    下面演示利用 HTTP 协议编写一个多线程下载本文源代码的程序,源代码地址为 https://files.cnblogs.com/liuning8023/HttpDownload.rar

    1,新建项目 "HttpDownload";

    2,Form1.Desginer.cs 代码如下所示:

    namespace HttpDownload
    {
        partial class Form1
        {
            /// <summary>
            /// 必需的设计器变量。
            /// </summary>
            private System.ComponentModel.IContainer components = null;
     
            /// <summary>
            /// 清理所有正在使用的资源。
            /// </summary>
            /// <param name="disposing">如果应释放托管资源,为 true;否则为 false。</param>
            protected override void Dispose(bool disposing)
            {
                if (disposing && (components != null))
                {
                    components.Dispose();
                }
                base.Dispose(disposing);
            }
     
            #region Windows 窗体设计器生成的代码
     
            /// <summary>
            /// 设计器支持所需的方法 - 不要
            /// 使用代码编辑器修改此方法的内容。
            /// </summary>
            private void InitializeComponent()
            {
                this.lst_processing = new System.Windows.Forms.ListBox();
                this.lbl_url = new System.Windows.Forms.Label();
                this.lbl_localFile = new System.Windows.Forms.Label();
                this.lbl_threadNum = new System.Windows.Forms.Label();
                this.txt_url = new System.Windows.Forms.TextBox();
                this.txt_localFile = new System.Windows.Forms.TextBox();
                this.txt_threadNum = new System.Windows.Forms.TextBox();
                this.btn_rec = new System.Windows.Forms.Button();
                this.txt_overTime = new System.Windows.Forms.TextBox();
                this.lbl_overTime = new System.Windows.Forms.Label();
                this.SuspendLayout();
                // 
                // lst_processing
                // 
                this.lst_processing.FormattingEnabled = true;
                this.lst_processing.ItemHeight = 12;
                this.lst_processing.Location = new System.Drawing.Point(12, 12);
                this.lst_processing.Name = "lst_processing";
                this.lst_processing.Size = new System.Drawing.Size(342, 364);
                this.lst_processing.TabIndex = 0;
                // 
                // lbl_url
                // 
                this.lbl_url.AutoSize = true;
                this.lbl_url.Location = new System.Drawing.Point(377, 17);
                this.lbl_url.Name = "lbl_url";
                this.lbl_url.Size = new System.Drawing.Size(47, 12);
                this.lbl_url.TabIndex = 1;
                this.lbl_url.Text = "文件URL";
                // 
                // lbl_localFile
                // 
                this.lbl_localFile.AutoSize = true;
                this.lbl_localFile.Location = new System.Drawing.Point(377, 42);
                this.lbl_localFile.Name = "lbl_localFile";
                this.lbl_localFile.Size = new System.Drawing.Size(53, 12);
                this.lbl_localFile.TabIndex = 2;
                this.lbl_localFile.Text = "本地地址";
                // 
                // lbl_threadNum
                // 
                this.lbl_threadNum.AutoSize = true;
                this.lbl_threadNum.Location = new System.Drawing.Point(377, 74);
                this.lbl_threadNum.Name = "lbl_threadNum";
                this.lbl_threadNum.Size = new System.Drawing.Size(41, 12);
                this.lbl_threadNum.TabIndex = 3;
                this.lbl_threadNum.Text = "线程数";
                // 
                // txt_url
                // 
                this.txt_url.Location = new System.Drawing.Point(450, 17);
                this.txt_url.Name = "txt_url";
                this.txt_url.Size = new System.Drawing.Size(353, 21);
                this.txt_url.TabIndex = 4;
                this.txt_url.Text = "https://files.cnblogs.com/liuning8023/HttpDownload.rar";
                // 
                // txt_localFile
                // 
                this.txt_localFile.Location = new System.Drawing.Point(450, 44);
                this.txt_localFile.Name = "txt_localFile";
                this.txt_localFile.Size = new System.Drawing.Size(232, 21);
                this.txt_localFile.TabIndex = 5;
                this.txt_localFile.Text = "c:////download.rar";
                // 
                // txt_threadNum
                // 
                this.txt_threadNum.Location = new System.Drawing.Point(450, 71);
                this.txt_threadNum.Name = "txt_threadNum";
                this.txt_threadNum.Size = new System.Drawing.Size(232, 21);
                this.txt_threadNum.TabIndex = 6;
                this.txt_threadNum.Text = "5";
                // 
                // btn_rec
                // 
                this.btn_rec.Location = new System.Drawing.Point(607, 137);
                this.btn_rec.Name = "btn_rec";
                this.btn_rec.Size = new System.Drawing.Size(75, 23);
                this.btn_rec.TabIndex = 7;
                this.btn_rec.Text = "接收";
                this.btn_rec.UseVisualStyleBackColor = true;
                this.btn_rec.Click += new System.EventHandler(this.btn_rec_Click);
                // 
                // txt_overTime
                // 
                this.txt_overTime.Location = new System.Drawing.Point(450, 99);
                this.txt_overTime.Name = "txt_overTime";
                this.txt_overTime.Size = new System.Drawing.Size(232, 21);
                this.txt_overTime.TabIndex = 8;
                // 
                // lbl_overTime
                // 
                this.lbl_overTime.AutoSize = true;
                this.lbl_overTime.Location = new System.Drawing.Point(377, 102);
                this.lbl_overTime.Name = "lbl_overTime";
                this.lbl_overTime.Size = new System.Drawing.Size(53, 12);
                this.lbl_overTime.TabIndex = 9;
                this.lbl_overTime.Text = "结束时间";
                // 
                // Form1
                // 
                this.AutoScaleDimensions = new System.Drawing.SizeF(6F, 12F);
                this.AutoScaleMode = System.Windows.Forms.AutoScaleMode.Font;
                this.ClientSize = new System.Drawing.Size(815, 384);
                this.Controls.Add(this.lbl_overTime);
                this.Controls.Add(this.txt_overTime);
                this.Controls.Add(this.btn_rec);
                this.Controls.Add(this.txt_threadNum);
                this.Controls.Add(this.txt_localFile);
                this.Controls.Add(this.txt_url);
                this.Controls.Add(this.lbl_threadNum);
                this.Controls.Add(this.lbl_localFile);
                this.Controls.Add(this.lbl_url);
                this.Controls.Add(this.lst_processing);
                this.Name = "Form1";
                this.Text = "Form1";
                this.ResumeLayout(false);
                this.PerformLayout();
     
            }
     
            #endregion
     
            public System.Windows.Forms.ListBox lst_processing;
            private System.Windows.Forms.Label lbl_url;
            private System.Windows.Forms.Label lbl_localFile;
            private System.Windows.Forms.Label lbl_threadNum;
            private System.Windows.Forms.TextBox txt_url;
            private System.Windows.Forms.TextBox txt_localFile;
            private System.Windows.Forms.TextBox txt_threadNum;
            private System.Windows.Forms.Button btn_rec;
            private System.Windows.Forms.TextBox txt_overTime;
            private System.Windows.Forms.Label lbl_overTime;
        }
    }

    说明:

    1)  添加四个 Lable 控件和 TextBox 控件;一个 ListBox 控件;一个 Button 控件;

    2)  ListBox 控件需要将 private 属性改为 public,以便在外部使用。

    3,新建 HttpMultiThreadDownload.cs 类,代码如下:

    using System;
    using System.Net;
    using System.IO;
    using System.Windows.Forms;
     
    namespace HttpDownload
    {
        /// <summary>
        /// 调用外部窗体
        /// </summary>
        /// <param name="text"></param>
        delegate void ProcessingCallback(string processing);
     
        public class HttpMultiThreadDownload
        {
            const int _bufferSize = 512;
            public Form1 frm { get; set; }
            public int ThreadId { get; set; }             // 线程 ID                 
            public string Url { get; set; }               // 文件 Url
     
            public HttpMultiThreadDownload(Form1 form, int threadId)
            {
                frm = form;
                ThreadId = threadId;
                Url = frm.Url;
            }
            /// <summary>
            /// 析构方法
            /// </summary>
            ~HttpMultiThreadDownload()
            {
                if (!frm.InvokeRequired)
                {
                    frm.Dispose();
                }
            }
            /// <summary>
            /// 接收
            /// </summary>
            public void receive()
            {
                string filename = frm.fileNames[ThreadId];     // 线程临时文件
                byte[] buffer = new byte[_bufferSize];         // 接收缓冲区
                int readSize = 0;                              // 接收字节数
                FileStream fs = new FileStream(filename, System.IO.FileMode.Create);
                Stream ns = null;
     
                this.SetListBox("线程[" + ThreadId.ToString() + "] 开始接收......");
                try
                {
                    HttpWebRequest request = (HttpWebRequest)WebRequest.Create(Url);
                    request.AddRange(frm.StartPos[ThreadId], frm.StartPos[ThreadId] + frm.fileSize[ThreadId]);
                    ns = request.GetResponse().GetResponseStream();
                    readSize = ns.Read(buffer, 0, _bufferSize);
                    this.SetListBox("线程[" + ThreadId.ToString() + "] 正在接收 " + readSize);
                    while (readSize > 0)
                    {
                        fs.Write(buffer, 0, readSize);
                        readSize = ns.Read(buffer, 0, _bufferSize);
                        this.SetListBox("线程[" + ThreadId.ToString() + "] 正在接收 " + readSize);
                    }
                    fs.Close();
                    ns.Close();
                }
                catch (Exception er)
                {
                    MessageBox.Show(er.Message);
                    fs.Close();
                }
                this.SetListBox("进程[" + ThreadId.ToString() + "] 结束!");
                frm.threadStatus[ThreadId] = true;
            }
            private void SetListBox(string processing)
            {
                if (frm.lst_processing.InvokeRequired)
                {
                    ProcessingCallback d = new ProcessingCallback(SetListBox);
                    frm.Invoke(d, new object[] { processing });
                }
                else
                {
                    frm.lst_processing.Items.Add(processing);
                }
            }
        }
    }

    说明:

    1)该类使用了析构函数。通常,.NET Framework 垃圾回收器会隐式地管理对象的内存分配和释放。 但是,当应用程序封装窗口、文件和网络连接这类非托管资源时,应当使用析构函数释放这些资源。 当对象符合析构时,垃圾回收器将运行对象的 Finalize 方法。

    参考:http://msdn.microsoft.com/zh-cn/library/66x5fx1b.aspx

    2)另外,SetListBox 方法确保以线程安全方式访问 ListBox 控件。

    参考:http://msdn.microsoft.com/zh-cn/library/ms171728(v=VS.90).aspx

    3)多线程下载的关键在于能够定位下载文件中流的指定位置,例如 HttpWebRequest.AddRange 方法,从而让每个线程下载你指定的位置。该方式的下载类似于在浏览器中右键的“另存为”,不能用该方式下载类似 HTML 文件,因为这样的流是不能寻址,不能定位的。

    4,Form1.cs 代码如下所示:

    using System;
    using System.Net;
    using System.IO;
    using System.Windows.Forms;
     
    namespace HttpDownload
    {
        public partial class Form1 : Form
        {
            public int threadNum { get; set; }          // 进程
            public bool[] threadStatus { get; set; }    // 每个线程结束标志
            public string[] fileNames { get; set; }     // 每个线程接收文件的文件名
            public int[] StartPos { get; set; }     // 每个线程接收文件的起始位置
            public int[] fileSize { get; set; }         // 每个线程接收文件的大小
            public string Url { get; set; }             // 接受文件的URL
            public bool HasMerge { get; set; }           // 文件合并标志
     
            public Form1()
            {
                InitializeComponent();
            }
            /// <summary>
            ///
            /// </summary>
            /// <param name="sender"></param>
            /// <param name="e"></param>
            private void btn_rec_Click(object sender, EventArgs e)
            {
                Url = this.txt_url.Text.Trim().ToString();
                long fileSizeAll = 0;
     
                HttpWebRequest request = (HttpWebRequest)WebRequest.Create(this.Url);
                fileSizeAll = request.GetResponse().ContentLength;
                request.Abort();
                threadNum = int.Parse(this.txt_threadNum.Text.Trim().ToString());
                Init(fileSizeAll);
                for (int i = 0; i < threadNum; i++)
                {
                    this.lst_processing.Items.Add("线程[" + i + "]:" + (threadStatus[i] ? "结束" : "开始......"));
                    this.lst_processing.Items.Add(("开始位置:" + StartPos[i].ToString()).PadLeft(20, ' ') +
                        ("总大小:" + fileSize[i].ToString()).PadLeft(20, ' '));
                }
                this.lst_processing.Items.Add("------------------------------文件总大小:" + fileSizeAll);
     
                // 定义并启动线程数组
                System.Threading.Thread[] threads = new System.Threading.Thread[threadNum];
                HttpMultiThreadDownload[] httpDownloads = new HttpMultiThreadDownload[threadNum];
                for (int i = 0; i < threadNum; i++)
                {
                    httpDownloads[i] = new HttpMultiThreadDownload(this, i);
                    threads[i] = new System.Threading.Thread(new System.Threading.ThreadStart(httpDownloads[i].receive));
                    threads[i].Start();
                }
                System.Threading.Thread merge = new System.Threading.Thread(new System.Threading.ThreadStart(MergeFile));
                merge.Start();
                this.txt_overTime.Text = DateTime.Now.ToString();
            }
            /// <summary>
            /// 初始化
            /// </summary>
            /// <remarks>
            /// 每个线程平均分配文件大小,剩余部分由最后一个线程完成
            /// </remarks>
            /// <param name="filesize"></param>
            private void Init(long filesize)
            {
                threadStatus = new bool[threadNum];
                fileNames = new string[threadNum];
                StartPos = new int[threadNum];
                fileSize = new int[threadNum];
                int filethread = (int)filesize / threadNum;
                int filethreade = filethread + (int)filesize % threadNum;
                for (int i = 0; i < threadNum; i++)
                {
                    threadStatus[i] = false;
                    fileNames[i] = i.ToString() + ".dat";
                    if (i < threadNum - 1)
                    {
                        StartPos[i] = filethread * i;
                        fileSize[i] = filethread - 1;
                    }
                    else
                    {
                        StartPos[i] = filethread * i;
                        fileSize[i] = filethreade - 1;
                    }
                }
            }
            /// <summary>
            /// 合并文件
            /// </summary>
            public void MergeFile()
            {
                while (true)
                {
                    HasMerge = true;
                    for (int i = 0; i < threadNum; i++)
                    {
                        if (threadStatus[i] == false) // 若有未结束线程,则等待
                        {
                            HasMerge = false;
                            System.Threading.Thread.Sleep(100);
                            break;
                        }
                    }
                    if (HasMerge == true) // 否则,停止等待
                        break;
                }
     
                int bufferSize = 512;
                int readSize;
                string downFileNamePath = txt_localFile.Text.Trim().ToString();
                byte[] bytes = new byte[bufferSize];
                FileStream fs = new FileStream(downFileNamePath, FileMode.Create);
                FileStream fsTmp = null;
     
                for (int k = 0; k < threadNum; k++)
                {
                    fsTmp = new FileStream(fileNames[k], FileMode.Open);
                    while (true)
                    {
                        readSize = fsTmp.Read(bytes, 0, bufferSize);
                        if (readSize > 0)
                            fs.Write(bytes, 0, readSize);
                        else
                            break;
                    }
                    fsTmp.Close();
                }
                fs.Close();
                MessageBox.Show("接收完毕!!!");
            }
        }
    }

    程序运行结果如下图所示:

    2012-10-07_103453

    参考资料


    修改记录


    • 2012-10-07 [ADD][UPDATE]
    • 2012-10-08 [UPDATE]

    下载 Demo

  • 相关阅读:
    rest-assured : Restful API 测试利器
    Bootstrap 模态框 + iframe > 打开子页面 > 数据传输/关闭模态框
    angular js 自定义js错误处理(Angularjs js error handler)
    Spring Boot + Bootstrap 出现"Failed to decode downloaded font"和"OTS parsing error: Failed to convert WOFF 2.0 font to SFNT"
    Use Apache HttpClient to Post json data
    小程序-走迷宫
    Java getResourceAsStream() 方法会缓存文件的问题
    【转载】java项目中经常碰到的内存溢出问题: java.lang.OutOfMemoryError: PermGen space, 堆内存和非堆内存,写的很好,理解很方便
    DIV布局-高度不同DIV,自动换行并对齐
    Excel表格常用的函数,留着备用
  • 原文地址:https://www.cnblogs.com/liuning8023/p/2712097.html
Copyright © 2011-2022 走看看