TypeError: a bytes-like object is required, not 'str'
json.decoder.JSONDecodeError: Extra data
json文件太大,改为单行读取: with open("a", "r+") as f: data = json.load(f) 改为: with open("a.json", "r+") as f: data = f.readline()
UnicodeEncodeError: 'gbk' codec can't encode character u'xe7' in position
将 print(res.txt) 改成 print(res.text).encode('gbk','ignore')
Https相关
![](https://images.cnblogs.com/OutliningIndicators/ContractedBlock.gif)
1 1 requests发送HTTPS 2 2 前提:verify=False 3 3 报错:InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings 4 4 解决: 5 5 import urllib3 6 6 urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning) 7 7 8 8 多线程 - connectionpool.py https请求 InsecureRequestWarning 9 9 解决:注释掉connectionpool.py的845-851行
IDE相关
![](https://images.cnblogs.com/OutliningIndicators/ContractedBlock.gif)
1 Pycharm 2 全局查询关键字 3 按快捷键Ctrl + Shift + F或从从菜单Edit-》Find-》Find in Path进入全局查找界面
爬虫相关
Max retries exceeded with url 1.requests.adapters.DEFAULT_RETRIES = 5 2.s = requests.session() s.keep_alive = False
Pycharm
File size exceeds configured limit 2.5M. Code insight features not available help -》Edit custom properities -》输入 idea.max.intellisense.filesize=99999
字节/字符转换问题
TypeError: a bytes-like object is required, not ‘str’ str -》 encode()-》bytes bytes -》decode() -》str
共存
python3-m pip install --upgrade pip --force-reinstall
python2-m pip install --upgrade pip --force-reinstall
netaddr
用途
ip地址管理
安装
pip3 install netaddr
模块 - Queue
用途
队列
使用
from queue import Queue
包安装报错
Crypto.Cipher
![](https://img2018.cnblogs.com/common/1360174/201911/1360174-20191125162605086-48458987.png)
![](https://img2018.cnblogs.com/common/1360174/201911/1360174-20191125162503408-345868772.png)