zoukankan      html  css  js  c++  java
  • python 线程,进程28原则

    基于函数实现

    from threading import Thread
    
    
    def fun(data, *args, **kwargs):
        """
    
        :param data:
        :param args:
        :param kwargs:
        :return:
        """
        print('start %s'%data)
    
    
    if __name__ == "__main__":
        tups = ((1,),(2,),(3,))
        for i in tups:
            kwargs = {
                "target": fun,
                'args':i
            }
            t = Thread(**kwargs)
            t.start()  

    基于类实现

    from threading import Thread
    from multiprocessing import Process
    
    
    class MyThread(Thread):
        def __init__(self, func,*args):
            super(MyThread,self).__init__()
            self.func = func
            self.args = args
    
        def run(self):
            self.func(*self.args)
    
    
    class MyProcess(Process):
        def __init__(self, func,*args):
            super(MyProcess, self).__init__()
            self.func = func
            self.args = args
    
        def run(self):
            self.func(*self.args)
    

      

    传入类的入口函数,即可实现多线程

    baidu = Baidu()
    sogou = Sogou()
    google = Google()
    
    baiduThread = MyThread(baidu.run,1,100)
    sogouThread = MyThread(sogou.run,1,100)
    googleThread = MyThread(google.run,1,100)
    # 线程开启
    baiduThread.start()
    sogouThread.start()
    googleThread.start()
    # 主线程等待子线程
    baiduThread.join()
    sogouThread.join()
    googleThread.join()
    

      

    总结一波,实际实现多进程的进程是类的实例化对象的run函数(baidu.run),因此在初始化类时是串行的,而很多模块并不能在同一进程中实例化多个对象。

    例如:

    class MyClass:
        def __init__(self):
            logging.basicConfig(filename='yaomaitong.log', filemode="w", level=logging.INFO) # 典型的反例
    
        def run(self,*args):
            # 所有需要多进程的程序都该写在这里
            self.start()
            self.continue()
            self.end()
            pass
        
        def start(self):
            pass
    ...
    

      

    线程池、进程池:

    from concurrent.futures import ThreadPoolExecutor
    from concurrent.futures import ProcessPoolExecutor
    
    import requests
    
    
    urls_list = [
        'https://www.baidu.com',
        'http://www.gaosiedu.com',
        'https://www.jd.com',
        'https://www.taobao.com',
        'https://news.baidu.com',
    ]
    pool = ThreadPoolExecutor(3)
    
    def request(url):
        response = requests.get(url)
        return response
    
    def read_data(future,*args,**kwargs):
        response = future.result()
        response.encoding = 'utf-8'
        print(response.status_code,response.url)
    
    def main():
        for url in urls_list:
            done = pool.submit(request,url)
            done.add_done_callback(read_data)
    
    if __name__ == '__main__':
        main()
        pool.shutdown(wait=True)
    

      

  • 相关阅读:
    【MySQL】查看数据库所占空间大小
    Redis 和 Memcached 有什么区别?Redis 的线程模型是什么?为什么单线程的 Redis 比多线程的 Memcached 效率要高得多?
    Hbase 大表快速count
    redis击穿,穿透,雪崩以及解决方案
    elasticsearch和lucene的关系以及elasticsearch的核心概念
    Java 序列化 之 Serializable
    win10+virtualBox(CentOS7)java 环境搭建之 mysql8安装
    win10+virtualBox(CentOS7)java 环境搭建之 jdk 1.8安装
    win10+virtualBox(CentOS7)java 环境搭建之 网络设置
    joplin for Arch Linux
  • 原文地址:https://www.cnblogs.com/zenan/p/9188521.html
Copyright © 2011-2022 走看看