Python3-问题整理

TypeError: a bytes-like object is required, not 'str'

json.decoder.JSONDecodeError: Extra data

json文件太大，改为单行读取：

with open("a", "r+") as f:
data = json.load(f)

改为：

with open("a.json", "r+") as f:
data = f.readline()

UnicodeEncodeError: 'gbk' codec can't encode character u'xe7' in position

将 print(res.txt) 改成 print(res.text).encode('gbk','ignore')

Https相关

1 requests发送HTTPS
2 前提：verify=False
3 报错：InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
4 解决：
5 import urllib3
6 urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)
7 
8 多线程 - connectionpool.py https请求 InsecureRequestWarning
9 解决：注释掉connectionpool.py的845-851行

 

IDE相关

Pycharm
全局查询关键字
按快捷键Ctrl + Shift + F或从从菜单Edit-》Find-》Find in Path进入全局查找界面

 

爬虫相关

Max retries exceeded with url
1.requests.adapters.DEFAULT_RETRIES = 5
2.s = requests.session()
   s.keep_alive = False

 

Pycharm

File size exceeds configured limit 2.5M. Code insight features not available
help -》Edit custom properities -》输入 idea.max.intellisense.filesize=99999

 

字节/字符转换问题

TypeError: a bytes-like object is required, not ‘str’
str -》 encode()-》bytes
bytes -》decode() -》str

 

共存

python3-m pip install --upgrade pip --force-reinstall 
python2-m pip install --upgrade pip --force-reinstall 

netaddr

用途
ip地址管理

安装
pip3 install netaddr

模块 - Queue

用途
队列

使用
from queue import Queue

包安装报错

Crypto.Cipher