杂记
Typora 上传图片到博客园
import requests
import json
from sys import argv
uploadUrl = 'https://upload.cnblogs.com/imageuploader/processupload?host=www.cnblogs.com'
headers = {
"user-agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.149 Safari/537.36",
"cookie": "__gads=ID=0df8844e3171cf5d:T=1626703371:S=ALNI_MbCQzZlPNaCt56Y3ZVBMgBgxo-LiA; UM_distinctid=17b02249c8337a-021de176cab159-3f3a5e08-1fa400-17b02249c84437; _ga_3Q0DVSGN10=GS1.1.1629294955.2.1.1629294963.0; _ga=GA1.2.1874840476.1626703371; .Cnblogs.AspNetCore.Cookies=CfDJ8NACB8VE9qlHm6Ujjqxvg5A9JzQnWwTQwllXBS3jsuzbCTLy2gQscie0pLfzrbMgn704PWOBvn29tPVe9SrWwTZDRV8hTTISIxYAGXMLFead8T1oy-hTGkXxDwlskdW8yHcTMpvI481DuI0BGX4LeP5jib45Xj6vGjzibGkPiBou2eNyGQt3tVlMbHaSKcAspkeQmALgMVJEB-9ZoiijWgHo8M3_6F7RIUNgXml1yjrhu9bvWHeD6-tRHOigQRAQjS4QBG0hP_LXMT7RcXKFdy8AE2ZuCqTNZzrrNjwqXM7TpNw7uXpo2t0lo7tJ68MwJsoQPDkV3XIBIqKXyhc7bbzkm-OxcdlBNseZaxVm0gge1oRIIOo8NZoWePmQjjfF2TmwHgpdoZfH6kkc1FLCsB8THIUuAQGee9p7iK3Gx1CEmAfhILRECIwPxE0wxPdjl6FWvIANUJ8rELfJfUWKfxhVhhU3So1Ywlgv9tPTKegzgUSooWyzn7Fnlb4cLkGrXM4ncoAHUOFzwpslnrwS4vp3-12wXeOb2ggb7NfVL3TlEPojFPPj1hHDRTkeOfGo9A; _gid=GA1.2.205401666.1631025338; affinity=1631273060.452.42.609693"
}
# 类型映射
mimeMapping = {".png": 'image/png', '.gif': 'image/gif', '.jpg': 'image/jpeg', '.jpeg': 'image/jpeg'}
for i in argv[1:]:
# 图片地址参数
imgPath = i
# 对应的mime
mime = imgPath[imgPath.rindex("."):]
file = [
("",("fileName", open(imgPath, "rb"), mimeMapping[mime]))
]
response = requests.post(uploadUrl,headers = headers,files = file)
data = json.loads(response.text)
print(data['message'])
偏好设置中需要配置为上传服务
Flink on Yarn 集群报错
报错内容
Exception in thread "Thread-5" java.lang.IllegalStateException: Trying to access closed classloader. Please check if you store classloaders directly or indirectly in static fields. If the stacktrace suggests that the leak occurs in a third party library and cannot be fixed immediately, you can disable this check with the configuration 'classloader.check-leaked-classloader'.
at org.apache.flink.runtime.execution.librarycache.FlinkUserCodeClassLoaders$SafetyNetWrapperClassLoader.ensureInner(FlinkUserCodeClassLoaders.java:164)
at org.apache.flink.runtime.execution.librarycache.FlinkUserCodeClassLoaders$SafetyNetWrapperClassLoader.getResource(FlinkUserCodeClassLoaders.java:183)
at org.apache.hadoop.conf.Configuration.getResource(Configuration.java:2737)
at org.apache.hadoop.conf.Configuration.getStreamReader(Configuration.java:2993)
at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2952)
at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2925)
at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2805)
at org.apache.hadoop.conf.Configuration.get(Configuration.java:1199)
at org.apache.hadoop.conf.Configuration.getTimeDuration(Configuration.java:1787)
at org.apache.hadoop.util.ShutdownHookManager.getShutdownTimeout(ShutdownHookManager.java:183)
at org.apache.hadoop.util.ShutdownHookManager.shutdownExecutor(ShutdownHookManager.java:145)
at org.apache.hadoop.util.ShutdownHookManager.access$300(ShutdownHookManager.java:65)
at org.apache.hadoop.util.ShutdownHookManager$1.run(ShutdownHookManager.java:102)
解决方案
在conf/flink-conf.yaml,大约在192行添加
classloader.check-leaked-classloader: false
重新执行 Flink per-job cluster 调用WordCount样例
bin/flink run -m yarn-cluster -yjm 1024 -ytm 1024 examples/batch/WordCount.jar
正常打印输出结果.
Flink On Session 模式
- 先创建一个on Yarn的Flink 容器
bin/yarn-session.sh -jm 1024m -tm 1024m -d
执行测试PI脚本
bin/flink run ./examples/batch/WordCount.jar
查看默认配置信息
more /tmp/.yarn-properties-root
杀掉yarn上的Flink 容器
yarn application -kill application_1631228277006_0008
