zoukankan      html  css  js  c++  java
  • 淘宝IP地址库采集器c#代码

    这篇文章主要介绍了淘宝IP地址库采集器c#代码,有需要的朋友可以参考一下。
     最近做一个项目,功能类似于CNZZ站长统计功能,要求显示Ip所在的省份市区/提供商等信息。
    网上的Ip纯真数据库,下载下来一看,发现没提供商内容,省市区都很少,居然有XXX网吧,哥瞬间倒了。没标准化、并且杂乱、还不连续的IP段、总体说来没达到要求。

      在百度上找啊找,找到淘宝Ip地址库,官方介绍的相当诱人,准确率高,数据质量有保障,提供国家、省、市、县、运营商全方位信息,信息维度广,格式规范,但是限制每秒10次的访问(这个比较无语)。

      淘宝IP地址库,提供API    http://ip.taobao.com/

    接口说明

      1. 请求接口(GET): 
    http://ip.taobao.com/service/getIpInfo.php?ip=[ip地址字串] 
    2. 响应信息: 
    (json格式的)国家 、省(自治区或直辖市)、市(县)、运营商 
    3. 返回数据格式: 
    {"code":0,"data":{"ip":"210.75.225.254","country":"u4e2du56fd","area":"u534eu5317", 
    "region":"u5317u4eacu5e02","city":"u5317u4eacu5e02","county":"","isp":"u7535u4fe1", 
    "country_id":"86","area_id":"100000","region_id":"110000","city_id":"110000", 
    "county_id":"-1","isp_id":"100017"}}

          其中code的值的含义为,0:成功,1:失败。

    1 :IP转换

      准备好工具,后面就好弄啦, IPHelper提供了各种,IP<->byte[]<->Long 转换

    复制代码代码如下:

    public class IPHelper
            {
            /// <summary>
            /// ip转成long
            /// </summary>
            /// <param name="ip"></param>
            /// <returns></returns>
            public static long Ip2Long(string ip)
            {
                byte[] bytes = Ip2Bytes(ip);
                return Bytes2Long(bytes);
            }
            /// <summary>
            /// long转成ip
            /// </summary>
            /// <param name="ipLong"></param>
            /// <returns></returns>
            public static string Long2Ip(long ipLong)
            {
                byte[] bytes = Long2Bytes(ipLong);
                return Bytes2Ip(bytes);
            }
            /// <summary>
            /// long转成byte[]
            /// </summary>
            /// <param name="ipvalue"></param>
            /// <returns></returns>
            public static byte[] Long2Bytes(long ipvalue)
            {
                byte[] b = new byte[4];
                for (int i = 0; i < 4; i++)
                {
                    b[3 - i] = (byte)(ipvalue >> 8 * i & 255);
                }
                return b;
            }
            /// <summary>
            /// byte[]转成long
            /// </summary>
            /// <param name="bt"></param>
            /// <returns></returns>
            public static long Bytes2Long(byte[] bt)
            {
                int x = 3;
                long o = 0;
                foreach (byte f in bt)
                {
                    o += (long)f << 8 * x--;
                }
                return o;
            }
            /// <summary>
            /// ip转成byte[]
            /// </summary>
            /// <param name="ip"></param>
            /// <returns></returns>
            public static byte[] Ip2Bytes(string ip)
            { // http://www.jbxue.com
                string[] sp = ip.Split('.');
                return new byte[] { Convert.ToByte(sp[0]), Convert.ToByte(sp[1]), Convert.ToByte(sp[2]), Convert.ToByte(sp[3]) };
            }
            /// <summary>
            /// byte[]转成ip
            /// </summary>
            /// <param name="bytes"></param>
            /// <returns></returns>
            public static string Bytes2Ip(byte[] bytes)
            {
                return string.Format("{0}.{1}.{2}.{3}"
                                       , bytes[0]
                                       , bytes[1]
                                       , bytes[2]
                                       , bytes[3]);
            }
        }

    2 :多线程疯狂获取IP

    复制代码代码如下:

    /// <summary>
            /// 描述:开始采集
            /// </summary>
            private void StratCollect()
            {
                foreach (Thread thread in ThreadList)
                {
                    thread.Start();
                }
            }
            /// <summary>
            /// 描述:获取要采集的ip long
            /// </summary>
            private long GetCurrentIp()
            {
                long curip = System.Threading.Interlocked.Increment(ref CurrentCollectIP);
                return curip;
            }
            /// <summary>
            /// 线程中采集的方法
            /// </summary>
            private void GetTaobaoData()
            {
                long currentipLong = GetCurrentIp();
                while (currentipLong <= EndIP)
                {
                    try
                    {
                        CaptureTaobaoIPData(currentipLong);
                    }
                    catch (Exception ex)
                    { // http://www.jbxue.com
                          TextLog.SetString(currentipLong + ex.Message);
                    }
                    currentipLong = GetCurrentIp();
                }
            }
            /// <summary>
            /// 描述:线程中采集并得到IP
            /// </summary>
            private void CaptureTaobaoIPData(long currentipLong)
            {
                string ip = IPHelper.Long2Ip(currentipLong);
                string url = string.Format(UrlFomat, ip);
                string js =HttpHelper. HttpRequest(url, Encoding.UTF8);
                taobaoIPdata m = Newtonsoft.Json.JsonConvert.DeserializeObject<TaobaoJsonData>(js).data;
                m.ipLong = currentipLong;
                //更新界面
                this.Invoke(new Action<taobaoIPdata>(v =>
                {
                    taobaoIPdataList.Add(v);
                    this.dgv.DataSource = taobaoIPdataList;
                }), m);

            }

    3: Http请求的Json结果,并反序列化成对象

           http请求这个相当简单。网上一大把,这里主要说一下json序列化,在这里本人建议采用Newtonsoft.Json.dll 下载地址: http://json.codeplex.com/ 性能和兼容性达到最好

    Http请求

    复制代码代码如下:

    public class HttpHelper
        {
            public static string HttpRequest(string url, Encoding encoding)
            {
                try
                {
                    HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
                    request.Timeout = 6 * 1000;
                    request.Method = "GET";
                    //得到处理结果                              
                    HttpWebResponse response = (HttpWebResponse)request.GetResponse();
                    Stream myResponseStream = response.GetResponseStream();
                    StreamReader myStreamReader = new StreamReader(myResponseStream, encoding);
                    string result = myStreamReader.ReadToEnd();
                    return result;
                }
                catch (Exception ex)
                {
                    throw ex;
                }

            }

        }

    Json序列化

    复制代码代码如下:

    taobaoIPdata m = Newtonsoft.Json.JsonConvert.DeserializeObject<TaobaoJsonData>(js).data;

    序列号对象taobaoIPdata
    复制代码代码如下:

    /// <summary>
        /// 淘宝数据
        /// </summary>
        public partial class taobaoIPdata 
        {
            private long _ipLong;
            /// <summary>
            /// IP 长整形
            /// </summary>
            public long ipLong
            {
                get { return _ipLong; }
                set { _ipLong = value; }
            }

            private string _ip;
            /// <summary>
            /// IP地址
            /// </summary>
            public string ip
            { // http://www.jbxue.com
                 get { return _ip; }
                set { _ip = value; }
            }

            private string _country;
            /// <summary>
            /// 国家
            /// </summary>
            public string country
            {
                get { return _country; }
                set { _country = value; }
            }

            private string _country_id;
            /// <summary>
            /// 国家编号
            /// </summary>
            public string country_id
            {
                get { return _country_id; }
                set { _country_id = value; }
            }

            private string _area;
            /// <summary>
            /// 地区
            /// </summary>
            public string area
            {
                get { return _area; }
                set { _area = value; }
            }

            private string _area_id;
            /// <summary>
            /// 地区编号
            /// </summary>
            public string area_id
            {
                get { return _area_id; }
                set { _area_id = value; }
            }

            private string _region;
            /// <summary>
            /// 区域
            /// </summary>
            public string region
            {
                get { return _region; }
                set { _region = value; }
            }

            private string _region_id;
            /// <summary>
            /// 区域编号
            /// </summary>
            public string region_id
            {
                get { return _region_id; }
                set { _region_id = value; }
            }

            private string _city;
            /// <summary>
            ///城市
            /// </summary>
            public string city
            {
                get { return _city; }
                set { _city = value; }
            }

            private string _city_id;
            /// <summary>
            /// 城市编号
            /// </summary>
            public string city_id
            {
                get { return _city_id; }
                set { _city_id = value; }
            }

            private string _county;
            /// <summary>
            /// 县
            /// </summary>
            public string county
            {
                get { return _county; }
                set { _county = value; }
            }

            private string _county_id;
            /// <summary>
            /// 县编号
            /// </summary>
            public string county_id
            {
                get { return _county_id; }
                set { _county_id = value; }
            }

            private string _isp;
            /// <summary>
            /// 供应商
            /// </summary>
            public string isp
            {
                get { return _isp; }
                set { _isp = value; }
            }

            private string _isp_id;
            /// <summary>
            /// 供应商ID
            /// </summary>
            public string isp_id
            {
                get { return _isp_id; }
                set { _isp_id = value; }
            }

            

        }
        /// <summary>
        /// 淘宝api 返回的json数据
        /// </summary>
        public partial class TaobaoJsonData
        {
            public int code { get; set; }
            public taobaoIPdata data { get; set; }
        }

    序列号对象taobaoIPdata

    4:插入到数据库中。。。剩下的自己随便搞啦

  • 相关阅读:
    splay
    开车旅行(2012day1T3)
    LCT入门
    最小瓶颈路
    poj 3041 Asteroids
    sql waitfor 延时执行
    [Microsoft][ODBC SQL Server Driver][DBNETLIB]SQL Server 不存在或访问被拒绝
    SQL Server中行列转换
    sql中 with rollup 、with cube、grouping 统计函数用法
    sql 分组后 组内排名
  • 原文地址:https://www.cnblogs.com/linuxnotes/p/3484776.html
Copyright © 2011-2022 走看看