zoukankan      html  css  js  c++  java
  • 使用monkey技术修改python requests模块

    例如请求前和请求后各来一条日志,这样就不需要在自己的每个代码都去加日志了。

    其实也可以直接记录'urllib3.connectionpool'  logger name的日志。

    
    
    修改了requests,所有requests的地方将自动打印日志,前提要给custom_request添加handlers

    import
    requests from app.utils.utils_ydf import LogManager logger = LogManager('custom_request').get_logger_without_handlers() class CustomSession(requests.Session): def request(self, method, url, params=None, data=None, headers=None, cookies=None, files=None, auth=None, timeout=None, allow_redirects=True, proxies=None, hooks=None, stream=None, verify=None, cert=None, json=None): """Constructs a :class:`Request <Request>`, prepares it and sends it. Returns :class:`Response <Response>` object. :param method: method for the new :class:`Request` object. :param url: URL for the new :class:`Request` object. :param params: (optional) Dictionary or bytes to be sent in the query string for the :class:`Request`. :param data: (optional) Dictionary, bytes, or file-like object to send in the body of the :class:`Request`. :param json: (optional) json to send in the body of the :class:`Request`. :param headers: (optional) Dictionary of HTTP Headers to send with the :class:`Request`. :param cookies: (optional) Dict or CookieJar object to send with the :class:`Request`. :param files: (optional) Dictionary of ``'filename': file-like-objects`` for multipart encoding upload. :param auth: (optional) Auth tuple or callable to enable Basic/Digest/Custom HTTP Auth. :param timeout: (optional) How long to wait for the server to send data before giving up, as a float, or a :ref:`(connect timeout, read timeout) <timeouts>` tuple. :type timeout: float or tuple :param allow_redirects: (optional) Set to True by default. :type allow_redirects: bool :param proxies: (optional) Dictionary mapping protocol or protocol and hostname to the URL of the proxy. :param stream: (optional) whether to immediately download the response content. Defaults to ``False``. :param verify: (optional) Either a boolean, in which case it controls whether we verify the server's TLS certificate, or a string, in which case it must be a path to a CA bundle to use. Defaults to ``True``. :param cert: (optional) if String, path to ssl client cert file (.pem). If Tuple, ('cert', 'key') pair. :rtype: requests.Response """ # Create the Request. req = requests.Request( method=method.upper(), url=url, headers=headers, files=files, data=data or {}, json=json, params=params or {}, auth=auth, cookies=cookies, hooks=hooks, ) logger.debug('start {} this url --> {} '.format(method, url)) prep = self.prepare_request(req) proxies = proxies or {} settings = self.merge_environment_settings( prep.url, proxies, stream, verify, cert ) # Send the request. send_kwargs = { 'timeout': timeout, 'allow_redirects': allow_redirects, } send_kwargs.update(settings) resp = self.send(prep, **send_kwargs) logger.debug('{} {} {} {} {} {}'.format(method, resp.url, resp.status_code, resp.elapsed.total_seconds(), resp.is_redirect, resp.text.__len__())) return resp def custom_request(method, url, **kwargs): with CustomSession() as session: return session.request(method=method, url=url, **kwargs) def patch_request(): LogManager('custom_request').get_logger_and_add_handlers() requests.api.request = custom_request if __name__ == '__main__': patch_request() resp = requests.get('http://www.sina.com.cn') requests.get('https://www.baidu.com')

    1、结果就是这样。这样所有地方的requests请求都会生效了。

    正常情况下我有自己围绕Session类实例包装的SessionWrapper类,通常情况下我不会直接去使用requests的那些函数请求接口,但同事都是直接requests,然后每个地方打印print一下请求和返回,以便能看看到发了什么请求,那样有些重复。而且项目有几百个文件,突然蹦出来的print语句,根本不知道哪里print的。

    使用这种猴子技术后,不用修改库源码,就能生效。

    2、使用猴子技术要注意的是,patch的地方并不是随便替换就能生效的。

    如果库里面的那个地方是from xx import yy

    你直接yy.function = yourfunction

    这是无效的,一定要找到正确的地方替换,要搞清除到底怎么才能生效,必须要熟悉python的模块导入机制。可以看我的上上篇的三个试验文件。

    python 模块会导入几次?猴子补丁为什么可以实现?

  • 相关阅读:
    调度器2—cat /proc/<pid>/sched内容分析
    调度器1—相关接口和命令行工具
    Java中的String类
    Java中的数组
    代码访问使用Let's Encrypt证书的网站提示certificate has expired的解决方法
    Linux环境Clion使用Protobuf
    PyTorch Dataloader读取时如何在进程之间传输数据
    6 安装Grafana 展示promethues数据
    5 Prometheus relabel配置
    4 PromQL
  • 原文地址:https://www.cnblogs.com/ydf0509/p/9379559.html
Copyright © 2011-2022 走看看