C# 之 服务端获取远程资源

　　获取指定网页的远程资源可使用 WebClient、WebRequest、HttpWebRequest 三种方式来实现。当然也可使用webBrowse，webBrowse不做介绍。

　　通过 System.Net 名称空间下 WebClient 类下载文件。

        //通过WebClient下载网页的源码    
        string url = "http://www.example.com";
        System.Net.WebClient client = new System.Net.WebClient();
        client.Encoding = System.Text.Encoding.UTF8;
        string strHtml = client.DownloadString(url);

    

 综合实例：

using System;
using System.Text;

using System.IO;
using System.Net;

namespace SnapUtility
{
    public class ServerResourceHelper
    {
        /// <summary>
        /// 通过WebClient获取远程资源
        /// </summary>
        /// <param name="url"></param>
        /// <returns></returns>
        public static string GetByWebClient(string url)
        {
            string strReturn = null;
            WebClient client = new WebClient();
            //client.Headers.Add(HttpRequestHeader.Accept, @"text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8");
            //client.Headers.Add(HttpRequestHeader.Referer, null);
            //client.Headers.Add(HttpRequestHeader.UserAgent, @" Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/33.0.1750.154 Safari/537.36");
            //client.Headers.Add(HttpRequestHeader.ContentType, "application/octet-stream");
            
            string strExtension = Path.GetExtension(url); //扩展名
            if (String.IsNullOrEmpty(strExtension))
            {
                Stream stream = client.OpenRead(url);
                StreamReader streamReader = new StreamReader(stream, Encoding.GetEncoding("utf-8"));
                strReturn = streamReader.ReadToEnd();
                streamReader.Close();
                stream.Close();
            }
            else
            {
                string path = "../WebSnap/DownLoad/";
                path = System.Web.HttpContext.Current.Server.MapPath(path);

                //自动创建文件夹
                if (!Directory.Exists(path))
                {
                    Directory.CreateDirectory(path);
                }
                path += DateTime.Now.ToString("hhmmss") + strExtension;

                client.DownloadFile(new Uri(url), path);
            }
            client.Dispose();
            return strReturn;
        }

        /// <summary>
        /// 通过WebRequest获取远程资源
        /// </summary>
        /// <param name="url"></param>
        /// <returns></returns>
        public static string GetByWebRequest(string url)
        {
            string strReturn = null;
            Uri uri = new Uri(url);
            WebRequest request = WebRequest.Create(uri);
            WebResponse response = request.GetResponse();
            Stream stream = response.GetResponseStream();

            string strExtension = Path.GetExtension(url); //扩展名
            if (String.IsNullOrEmpty(strExtension))
            {
                StreamReader streamReader = new StreamReader(stream, Encoding.GetEncoding("utf-8"));
                strReturn = streamReader.ReadToEnd();
                streamReader.Close();
                stream.Close();
                response.Close();
            }
            else
            {
                string fileName = DateTime.Now.ToString("hhmmss") + strExtension;
                long length = response.ContentLength;
                strReturn = SaveFile(ref stream, fileName, length);
            }
            return strReturn;
        }

        /// <summary>
        /// 通过WebRequest的Http特定实现获取远程资源
        /// </summary>
        /// <param name="url">远程地址</param>
        /// <returns></returns>
        public static string GetByHttpWebRequest(string url)
        {
            string strReturn = null;
            Uri uri = new Uri(url);
            HttpWebRequest request = (HttpWebRequest)WebRequest.Create(uri);
            HttpWebResponse response = (HttpWebResponse)request.GetResponse();
            Stream stream = response.GetResponseStream();

            string strExtension = Path.GetExtension(url); //扩展名
            if (String.IsNullOrEmpty(strExtension))
            {
                StreamReader streamReader = new StreamReader(stream, System.Text.Encoding.GetEncoding("utf-8"));
                strReturn = streamReader.ReadToEnd();
                streamReader.Close();
                stream.Close();
                response.Close();
            }
            else
            {
                string fileName = DateTime.Now.ToString("hhmmss") + strExtension;
                long length = response.ContentLength;
                strReturn = SaveFile(ref stream, fileName, length);
            }
            return strReturn;
        }

        /// <summary>
        /// FileStream保存文件
        /// </summary>
        /// <param name="stream"></param>
        private static string SaveFile(ref Stream stream, string fileName, long length)
        {
            byte[] buffer = new byte[length];
            stream.Read(buffer, 0, buffer.Length);
            stream.Close();

            string path = AppDomain.CurrentDomain.BaseDirectory + "DownLoad\" + fileName;
            FileStream fileStream = new FileStream(path, FileMode.OpenOrCreate, FileAccess.Write);
            fileStream.Write(buffer, 0, buffer.Length);
            fileStream.Flush();
            fileStream.Close();

            return path;
        }

        /// <summary>
        /// 保存图片
        /// </summary>
        /// <param name="stream"></param>
        private static string SavePicture(ref Stream stream, string fileName)
        {
            System.Drawing.Image image = System.Drawing.Image.FromStream(stream);
            stream.Close();
            string path = "../WebSnap/DownLoad/";
            path = System.Web.HttpContext.Current.Server.MapPath(path);

            //自动创建文件夹
            if (!Directory.Exists(path))
            {
                Directory.CreateDirectory(path);
            }
            path += fileName;

            image.Save(path, System.Drawing.Imaging.ImageFormat.Png);
            image.Dispose(); //释放资源

            return path;
        }
    }
}

注意：

“utf-8”应与指定网页的编码对应。

可以看到HttpWebRequest 方式最复杂，但确提供了更多的选择性。

有的网站检测客户端的UserAgent！如163