zoukankan      html  css  js  c++  java
  • PHP并发之CURL

    <?php
    $start_time = microtime(true);
    $url = array();
    for ($i=0;$i<1000;$i++){
        $url[] = "http://www.baidu.com/s?wd=".$i;
    }
    $header = array();
    $result_array = multithread_request($url,$header);
    $end_time = microtime(true);
    
    echo $end_time - $start_time.PHP_EOL;
    
    function multithread_request($url,$headers)
    {
        //初始化一个curl批处理句柄资源  
        $mh = curl_multi_init();
        $headerArr = array();
        foreach( $headers as $n => $v ) {
            $headerArr[] = $n .':' . $v;
        }   
        foreach ($url as $i => $value) {   
          $conn[$i] = curl_init($url[$i]);
          curl_setopt($conn[$i], CURLOPT_CUSTOMREQUEST, "GET");
          curl_setopt($conn[$i], CURLOPT_SSL_VERIFYPEER, FALSE);
          curl_setopt($conn[$i], CURLOPT_SSL_VERIFYHOST, FALSE);
          curl_setopt($conn[$i], CURLOPT_HTTPHEADER, $headerArr);
          curl_setopt($conn[$i], CURLOPT_USERAGENT, 'Mozilla/5.0 (compatible; MSIE 5.01; Windows NT 5.0)');
          curl_setopt($conn[$i], CURLOPT_FOLLOWLOCATION, 1);
          curl_setopt($conn[$i], CURLOPT_AUTOREFERER, 1);
          //curl_setopt($conn[$i], CURLOPT_POST, true );
          //curl_setopt($conn[$i], CURLOPT_POSTFIELDS, $value);
          curl_setopt($conn[$i], CURLOPT_RETURNTRANSFER, true);
          curl_setopt($conn[$i], CURLOPT_HEADER ,0);
    //      curl_setopt($conn[$i], CURLOPT_CONNECTTIMEOUT,30);
          curl_setopt($conn[$i], CURLOPT_TIMEOUT, 50);  
    	  	curl_setopt($conn[$i], CURLOPT_HTTPPROXYTUNNEL, true);
          //log_msg('access',$value);   
          curl_multi_add_handle ($mh,$conn[$i]);
        }
    
        $active = null;
        do {
            $mrc = curl_multi_exec($mh, $active);
        } while ($mrc == CURLM_CALL_MULTI_PERFORM);
         
         
        while ($active and $mrc == CURLM_OK) {
            
            if(curl_multi_select($mh) === -1){
                usleep(100);
            }
            do {
                $mrc = curl_multi_exec($mh, $active);
            } while ($mrc == CURLM_CALL_MULTI_PERFORM);
         
        }
          
        $result_array = array();   
        foreach ($url as $i => $value) {   
          //返回错误
          if(curl_error($conn[$i]))
          {
             //log_msg('error',curl_error($conn[$i]));
          }
          //在设置了CURLOPT_RETURNTRANSFER的情况下,返回获取的输出的文本流
          $result_array[$i] = curl_multi_getcontent($conn[$i]); // 获得返回的数据   
          file_put_contents('curl/'.$i.'.html',$result_array[$i]);
        }   
           
        foreach ($url as $i => $value) {  
          curl_multi_remove_handle($mh,$conn[$i]);   
          curl_close($conn[$i]);   
        }   
           
        //关闭一个批处理句柄资源
        curl_multi_close($mh);
    
        return $result_array;   
    }
    
  • 相关阅读:
    【SQL】语句/函数汇总
    【SVN】提交报错:×××文件is not under version control
    【AnjularJS系列1】样式相关的指令
    【Javescript】DOM(文档对象模型)
    【HTML】框架集(Framesets)
    【HTML】<!DOCTYPE html>作用
    vue2.0移动端自定义性别选择提示框
    微信小程序踩坑记
    网页里如何使用js禁用F12事件
    网页里如何使用js屏蔽鼠标右击事件
  • 原文地址:https://www.cnblogs.com/yuzhoushenqi/p/7597252.html
Copyright © 2011-2022 走看看