zoukankan      html  css  js  c++  java
  • curl 采集的时候遇到301怎么办

    采集的时候遇到301,采集数据有错误

    $ch = curl_init($url);
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
    curl_setopt($ch, CURLOPT_BINARYTRANSFER,
    true);
    curl_setopt($ch
    , CURLOPT_HEADER, true);
    curl_setopt($ch,CURLOPT_ENCODING ,'gzip');
    curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE);
    curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, FALSE);
    curl_setopt($ch, CURLOPT_TIMEOUT, 20);
    curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);

    $header = array(
    "GET /product/detail/490725/invest/0/10?_=1477377497991 HTTP/1.1",
    "Host: www.xiaoniu88.com",
    "Connection: keep-alive",
    "Cache-Control: max-age=0",
    "Accept: application/json, text/javascript, */*; q=0.01",
    "X-Requested-With: XMLHttpRequest",
    "User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36",
    "Referer: http://www.xiaoniu88.com/product/planning/detail/490725",
    "Accept-Encoding: gzip, deflate, sdch",
    "Accept-Language: zh-CN,zh;q=0.8,en;q=0.6",
    'Cookie: sr=334.127.201.43.11.3.43.224.47.109.0.33.20.15.07; JSESSIONID=ABDA9A33E61801B02E7064AA6D50EFAF.t-9003; lcksid=59916c534e4cea0024d7691c; SESSIONID=38acb08a-32a5-4043-a9e5-8c52a48916f4; referer="https://www.xiaoniu88.com/user/_2017-08-14"; _gat=1; _ga=GA1.2.1713379385.1502698882; _gid=GA1.2.1557902220.1502698882; Hm_lvt_7226b8c48cd07619c7a9ebd471d9d589=1502698882; Hm_lpvt_7226b8c48cd07619c7a9ebd471d9d589=1502702891'
    );
    curl_setopt($ch, CURLOPT_HTTPHEADER, $header);
  • 相关阅读:
    spring cloud项目搭建
    获取iframe的window对象
    数学杂谈 #7
    [AGC023D] Go Home
    JOISC 2021 部分题解
    [NOI2017]泳池
    [NOI2016] 循环之美
    [NOI2016] 优秀的拆分
    [LG P3676]小清新数据结构题
    [ARC113F]Social Distance
  • 原文地址:https://www.cnblogs.com/matengfei123/p/7364671.html
Copyright © 2011-2022 走看看