zoukankan      html  css  js  c++  java
  • 如果数据量超过一万条,采取循环查询思路实例

    连锁查询
    $query = $query->from('paypal_transaction_list AS tl')
                      ->distinct('tl.transaction_updated_date')
    //                ->leftJoin('paypal_transaction_detail AS td','td.transaction_id','=','tl.transaction_id')
                    ->leftJoin('paypal_account AS a','a.paypal_account','=','tl.account_number')
                    ->select('a.email AS paypa1_email','tl.transaction_updated_date','tl.transaction_event_code','tl.transaction_status','tl.currency_code','tl.transaction_amount','tl.fee_amount','a.email','tl.transaction_id','tl.transaction_subject as item_name','tl.transaction_subject','tl.paypal_reference_id','tl.available_balance')
                    //->groupBy('tl.transaction_event_code', 'tl.transaction_amount')
                    ->orderBy('tl.transaction_updated_date','asc');
    
     if ($this->query) {
                $this->query($this->query);
            }
    
    
      /**
         * 数据库查询
         *
         * @return void
         */
        public function query(Builder $query, $size = 10000)
        {
            $page = 1;
            $size = $size ?: $this->size;
            do {
               //每次查询一万条,page自加+1,然后到最后数据不等于条数,就停止查询
                $datas = $query->skip(($page - 1) * $size)->take($size)->get();
                $this->array($datas);
                $page++;
            } while (count($datas) == $size);
        }
    
       /**
         * 数组处理
         *
         * @return void
         */
        public function array($data = [])
        {
            foreach ($data as $data) {
                $this->push($this->map($data));
            }
        }
    
     protected function map($data): array
        {
            return $data;
        }
    
    /**
         * 写入数据
         *
         * @param array $data
         * @return void
         */
        public function push($data)
        {
    
            $rows = [];
            foreach ($data as $value) {
                $rows[] = "	" . iconv('utf-8', 'GBK//IGNORE', (string) $value);
            }
            fputcsv($this->handle, $rows);
        }

     更新多条,变量不单独一次查出所有数据

     //查询出所有需要待更新的数据,分页处理
            $page = 1;
            $listingUpdate = new OnlineListingUpdate();
            $count = $listingUpdate->getCountByRawSql([$this->type], $itemId);
            // dd($count);
            $totalPages = ceil($count / $this->limit);
            do {
                $offset = ($page - 1) * $this->limit;
                $data = $listingUpdate->getListByRawSql([$this->type], $this->limit, $offset, $itemId);
    
                if (!empty($data)) {
                    foreach ($data as $row) {
                        $this->updateProduct($row);
                    }
                }
                $page++;
            } while ($page <= $totalPages);
  • 相关阅读:
    Lucene 基础理论
    .NET Micro Framework V4.2 QFE2新版本简介
    FlashPaper
    在django中实现QQ登录
    基于lucene的搜索服务器
    ASP.NET MVC的Razor引擎:RazorViewEngine
    .Net Micro Framework
    关于基于DDD+Event Sourcing设计的模型如何处理模型重构的问题的思考
    泛型
    Log4j源码分析及配置拓展
  • 原文地址:https://www.cnblogs.com/kevin-yang123/p/14383929.html
Copyright © 2011-2022 走看看