zoukankan      html  css  js  c++  java
  • 高性能分布式内存队列系统beanstalkd(转)

    beanstalkd 一个高性能、轻量级的分布式内存队列系统,最初设计的目的是想通过后台异步执行耗时的任务来降低高容量Web应用系统的页面访问延迟,支持过有9.5 million用户的Facebook Causes应用。后来开源,之后有PostRank大规模部署和使用,每天处理百万级任务.

    安装略了PHP客户端可使用pheanstalk. 可以去github上在看

    require_once('pheanstalk/pheanstalk_init.php');
    $pheanstalk = new Pheanstalk('127.0.0.1:11300');
    
    // Pushing things into the queue
    for($i=0; $i<1000; $i++) {   $job = new stdClass();   $job->envelope_id = rand();
      $job->date = date('Y-m-d H:i:s');
      $job_data = json_encode($job);
      $pheanstalk->useTube('test')->put($job_data);
      echo "pushed: " . $job_data . "
    ";
    }

    server.php 

    class Worker {
    
      private $path;
    
      public function __construct($path) {
        $this->setBasePath($path);
        $this->log('starting');
        require_once('pheanstalk/pheanstalk_init.php');
        $this->pheanstalk = new Pheanstalk('127.0.0.1:11300');
      }
    
      public function __destruct() {
        $this->log('ending');
      }
    
      private function setBasePath($path) {
        $this->path = $path;
      }
    
      public function run() {
        $this->log('starting to run');
        $cnt = 0;
        $done_jobs = array();
    
        while(1) {
          $job = $this->pheanstalk->watch('test')->ignore('default')->reserve();
          $job_encoded = json_decode($job->getData(), false);
          $done_jobs[] = $job_encoded;
          $this->log('job:'.print_r($job_encoded, 1));
          $this->pheanstalk->delete($job);
          $cnt++;
    
          $memory = memory_get_usage();
    
          $this->log('memory:' . $memory);
    
          if($memory > 1000000) {
            $this->log('exiting run due to memory limit');
            exit;
          }
    
          usleep(10);
        }
      }
    
      private function log($txt) {
        file_put_contents($this->path . '/log/worker.txt', $txt . "
    ", FILE_APPEND);
      }
    }
    
    Picking up things from the queue
    $worker = new Worker(dirname($argv[0]));
    $worker->run();

    注:
    1)可使用supervisor或deamontools等将php worker.php变为守护进程.
    2)其它语言类库参考

    worker端不一定用php来写,可以用Python 或是 nodejs ,go写,都比PHP强

  • 相关阅读:
    感悟.
    近期感悟
    android 新建项目无法自动生成R文件
    重装后各种碰壁
    rhapsody
    libevent
    ui android需要解决的问题
    sql对xml的解析
    将datarow转换为实体的方法
    网页中的一些我不熟的东西
  • 原文地址:https://www.cnblogs.com/jackluo/p/3728520.html
Copyright © 2011-2022 走看看