记录一下最近的笔记 redis的一些坑,老外总结得不错 比如使用scan/cursor方式来拉数据来替代keys命令 http://www.programmersought.com/article/89521650540/ https://redis.io/commands/scan 能用: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public Set<String> scan(String matchKey) { Set<String> keys = redisTemplate.execute((RedisCallback<Set<String>>) connection -> { Set<String> keysTmp = new HashSet<>(); Cursor<byte[]> cursor = connection.scan(new ScanOptions.ScanOptionsBuilder().match("*" + matchKey + "*").count(1000).build()); while (cursor.hasNext()) { keysTmp.add(new String(cursor.next())); } return keysTmp; }); return keys; } 我的代码成品, 对count的理解还是不深,待挖掘: public Set<String> scanKeys(String keyPattern) { Set<String> keys = (Set<String>) redisTemplate.execute((RedisCallback<Set<String>>) connection -> { Set<String> keysTmp = new HashSet<>(); ScanOptions scanOptions = new ScanOptions.ScanOptionsBuilder().match(keyPattern).build(); Cursor<byte[]> cursor = connection.scan(scanOptions); while (cursor.hasNext()) { keysTmp.add(new String(cursor.next())); } return keysTmp; }); return keys; } >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 主要还是安装es7的问题. https://blog.csdn.net/u011265001/article/details/100084335?depth_1-utm_source=distribute.pc_relevant.none-task-blog-BlogCommendFromMachineLearnPai2-1&utm_source=distribute.pc_relevant.none-task-blog-BlogCommendFromMachineLearnPai2-1 NG开启basic 认证 官方的文档 https://docs.nginx.com/nginx/admin-guide/security-controls/configuring-http-basic-authentication/ 老外写的, 不错~~~ https://www.tecmint.com/setup-nginx-basic-http-authentication/ yum install nginx, --->安装在了 /usr/sbin/nginx, 配置文件在/etc/nginx/中 htpasswd /etc/nginx/.htpasswd kibana tips: 这个工具的安装 # yum install httpd-tools [RHEL/CentOS] $ sudo apt install apache2-utils [Debian/Ubuntu] 然后交互式,输入密码, 就生成了密码文件 $ cat /xxxx/.htpasswd user1:$apr1$/woC1jnP$KAh0SsVn5qeSMjTtn0E9Q0 配置在nginx的配置文件中 location /api { auth_basic “Administrator’s Area”; auth_basic_user_file /etc/apache2/.htpasswd; } 就OK了. https://www.cnblogs.com/silent2012/p/8377837.html 可以参考 官网reverse proxy的配置在这里sample https://nginx.org/en/docs/http/ngx_http_proxy_module.html?_ga=2.27153012.815603832.1587469798-958214196.1587469798#example ############################################################################ docker问题解决, 同下: https://www.cnblogs.com/elson-zeng/p/12553329.html CentOS7 firewalld docker 端口映射问题，firewall开放端口后，还是不能访问，解决方案 # 宿主机ip: 192.168.91.19 docker run -itd --name tomcat -p 8080:8080 tomcat /usr/local/apache-tomcat-9.0.30/bin/startup.sh # 防火墙放开8080端口 firewall-cmd --add-port=8080/tcp --permanent # 问题：发现访问：192.168.91.19:8080 访问不通，关闭firewall后，又可以访问通了 # 解决方案，把docker0网卡添加到trusted域 firewall-cmd --permanent --zone=trusted --change-interface=br-d2aa50162455 # 重启加载配置 firewall-cmd --reload # firewall-cmd相关命令：https://www.cnblogs.com/Raodi/p/11625487.html 我的问题是, 命名防火墙禁用掉了9200端口, 别的机器还是可以访问到. 解决方法是, https://docs.docker.com/engine/reference/commandline/dockerd/#daemon-configuration-file #The default location of the configuration file on Linux is /etc/docker/daemon.json. #The --config-file flag can be used to specify a non-default location. docker的daemon.json里配置: "iptables": false --->把iptable禁用掉,--->直接导致,kibana无法访问elasticsearch了 然后再按上面博客, 把docker的一个区域加入了trusted里. --->kibana可以访问elasticsearch了. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 看来碰到问题的人好多 #跟我差不多 https://www.jianshu.com/p/69d3ab177655 https://www.jianshu.com/p/10c467600ef9 #docker service的配置文件在/usr/lib/systemd/system/docker.service #当时配置docker服务的代理 在这里 /etc/systemd/system/docker.service.d/http-proxy.conf 这里涉及到了3个配置文件: /etc/docker/daemon.json #守护进程各种配置, 配置项非常多, 包括docker镜像站点等 /usr/lib/systemd/system/docker.service #服务配置文件 /etc/systemd/system/docker.service.d/http-proxy.conf #配置代理 所以, 需要学习很多相关的知识, 算是还债. 因为一直没有搞得非常通透, 导致解决问题磕磕绊绊 1.systemctl命令相关, 启动脚本在哪里? 配置文件在哪里 2.systemctl daemon相关, 配置文件在哪里, 怎么配的 https://www.cnblogs.com/brucewhite/p/11608853.html systemctl --help 查看帮助!!!!!!! systemctl enable | disable | is-enabled | status | is-active unit systemctl get-default|set-default graphical.target|multi-user.target islate 在线切换模式 systemctl reload-daemon 加载新的unit 配置文件 systemd的unit 的配置文件位置:redhat系列： 目录:/usr/lib/systemd/system/ 下 enable 是在 /etc/systemd/system/multi-user.target.wants/ 这个目录下 做 unit 配置文件的软链： 3.linux自动启动的几种方式, init.d, chkconfig, service等等几个概念 https://www.cnblogs.com/liuxia912/p/10960610.html #CentOS 7的服务systemctl脚本存放在：/usr/lib/systemd/，有系统（system）和用户（user）之分， #像需要开机不登陆就能运行的程序，还是存在系统服务里吧，即：/usr/lib/systemd/system目录下 #对于那些支持 Systemd 的软件，安装的时候，会自动在/usr/lib/systemd/system目录添加一个配置文件。 在配置vnc服务器的时候也碰到了这个 systemctl daemon-reload >>待续 4.防火墙的几种配置, firewalld和iptables >>>>>>>>>>>>>>>>>>>>>>>>>>> $ firewall-cmd --zone=public --remove-port=10050/tcp ##这一句 runtime-to_permanent $ firewall-cmd --runtime-to-permanent $ firewall-cmd --reload >>>>>>>>>>>>>>>>>>>>>>>>>>> https://www.tecmint.com/start-stop-disable-enable-firewalld-iptables-firewall/ 下面几个课题有空再看 Understanding IPtables Firewall Basics and Tips Configure Iptables Firewall in Linux Configure FirewallD in Linux Useful FirewallD Rules to Manage Firewall in Linux How to Control Network Traffic Using FirewallD and Iptables # Also, you can mask the firewall service # which creates a symbolic link of the firewall.service to /dev/null, # thus disabling the service. # This is reverse of masking the service. # This removes the symlink of the service created during masking, # thus re-enabling the service. systemctl statrt/stop/restart/disable/enable/mask/unmask/status firewalld firewall-cmd --state systemctl restart firewalld.service(为什么要加一个.service, 是什么意思?) systemctl is-enabled firewalld.service #查看服务是否开机启动 systemctl list-unit-files|grep enabled #查看已启动的服务列表 systemctl --failed #查看启动失败的服务列表 关于iptables #How to Start/Stop and Enable/Disable IPtables Service systemctl start/stop/restart/disable/enable/status iptables service iptables start/stop/save/status #ubuntu/debian? sudo ufw enable/disable/status iptables -L -n -v >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 关于firewalld, 这个老外写得太好了! https://www.liquidweb.com/kb/an-introduction-to-firewalld/ firewall-cmd --permanent --add-port=22/TCP firewall-cmd --permanent --add-port=53/UDP firewall-cmd --permanent --remove-port=444/tcp firewall-cmd --permanent --add-service=ssh firewall-cmd --permanent --add-service=http firewall-cmd --permanent --remove-service=mysql #whitelist an ip address firewall-cmd --permanent --add-source=192.168.1.100 #以CIDR地址形式加一个范围, 这是最近学到的. firewall-cmd --permanent --add-source=192.168.1.0/24 firewall-cmd --permanent --remove-source=192.168.1.100 Block an IP Address #As the firewall-cmd tool is mostly used for opening or allowing access, #rich rules are needed to block an IP. #Rich rules are similar in form to the way iptables rules are written. firewall-cmd --permanent --add-rich-rule="rule family='ipv4' source address='192.168.1.100' reject" #CIDR range firewall-cmd --permanent --add-rich-rule="rule family='ipv4' source address='192.168.1.0/24' reject" Whitelist an IP Address for a Specific Port (More Rich Rules) firewall-cmd --permanent --add-rich-rule='rule family="ipv4" source address="192.168.1.100" port protocol="tcp" port="3306" accept' firewall-cmd --permanent --remove-rich-rule='rule family="ipv4" source address="192.168.1.100" port protocol="tcp" port="3306" accept' #save firewall-cmd --reload firewall-cmd --runtime-to-permanent #show firewall-cmd --list-all Bonus #1 :: Firewalld GUI Configuration You can find these under Applications > Sundry menu on CentOS 7.x and RedHat 7.x servers that have graphical access to. To install this application, from the command line, you can run the command: sudo yum install firewall-config Bonus #2 :: Firewalld Direct Rules /etc/firewalld/direct.xml >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> man 1 systemctl SYSTEMCTL(1) systemctl SYSTEMCTL(1) NAME systemctl - Control the systemd system and service manager SYNOPSIS systemctl [OPTIONS...] COMMAND [NAME...] DESCRIPTION systemctl may be used to introspect and control the state of the "systemd" system and service manager. Please refer to systemd(1) for an introduction into the basic concepts and functionality this tool manages. >>>加固Linux https://www.liquidweb.com/kb/security-for-your-linux-server/ >>>>> 中文的在这里, 比较冗长, 没有好好排版, 内容不错. https://www.linuxidc.com/Linux/2018-01/150072.htm firewall daemon 独立于 system-config-firewall，但二者不能同时使用。 如果你想使用自己的 iptables 和 ip6tables 静态防火墙规则, 那么请安装 iptables-services 并且禁用 firewalld ， 启用 iptables 和ip6tables: yum install iptables-services systemctl mask firewalld.service systemctl enable iptables.service systemctl enable ip6tables.service 静态防火墙规则配置文件是 /etc/sysconfig/iptables 以及 /etc/sysconfig/ip6tables . 什么是区域？ 网络区域定义了网络连接的可信等级。这是一个一对多的关系，这意味着一次连接可以仅仅是一个区域的一部分， 而一个区域可以用于很多连接。 >>> https://www.cnblogs.com/Raodi/p/11625487.html#_label1 >>>>>>>>>>>>>>>>>>>>>>>> 若 Nginx 收到请求为 https://ngxin_server_name/hello/world 而 Nginx 代理的路径为 /hello/（即在 location /hello/ 内设置代理） 则不以 / 结尾的被代理服务器收到的请求路径是 /hello/world 以 / 结尾的被代理服务器收到的请求路径是 /world 如果是为了在同一个域名下以不同路径分配不同的APP应选择后者以 / 结尾 原文链接：https://blog.csdn.net/randomparty/java/article/details/80961189
记录一下最近的笔记
redis的一些坑,老外总结得不错比如使用scan/cursor方式来拉数据来替代keys命令http://www.programmersought.com/article/89521650540/https://redis.io/commands/scan能用:>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public Set<String> scan(String matchKey) { Set<String> keys = redisTemplate.execute((RedisCallback<Set<String>>) connection -> { Set<String> keysTmp = new HashSet<>(); Cursor<byte[]> cursor = connection.scan(new ScanOptions.ScanOptionsBuilder().match("*" + matchKey + "*").count(1000).build()); while (cursor.hasNext()) { keysTmp.add(new String(cursor.next())); } return keysTmp; }); return keys; }我的代码成品, 对count的理解还是不深,待挖掘: public Set<String> scanKeys(String keyPattern) { Set<String> keys = (Set<String>) redisTemplate.execute((RedisCallback<Set<String>>) connection -> { Set<String> keysTmp = new HashSet<>(); ScanOptions scanOptions = new ScanOptions.ScanOptionsBuilder().match(keyPattern).build(); Cursor<byte[]> cursor = connection.scan(scanOptions); while (cursor.hasNext()) { keysTmp.add(new String(cursor.next())); } return keysTmp; }); return keys; }>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
主要还是安装es7的问题.https://blog.csdn.net/u011265001/article/details/100084335?depth_1-utm_source=distribute.pc_relevant.none-task-blog-BlogCommendFromMachineLearnPai2-1&utm_source=distribute.pc_relevant.none-task-blog-BlogCommendFromMachineLearnPai2-1
NG开启basic 认证官方的文档https://docs.nginx.com/nginx/admin-guide/security-controls/configuring-http-basic-authentication/老外写的, 不错~~~https://www.tecmint.com/setup-nginx-basic-http-authentication/
yum install nginx, --->安装在了 /usr/sbin/nginx, 配置文件在/etc/nginx/中
htpasswd /etc/nginx/.htpasswd kibanatips: 这个工具的安装# yum install httpd-tools[RHEL/CentOS]$ sudo apt install apache2-utils[Debian/Ubuntu]
然后交互式,输入密码, 就生成了密码文件$ cat /xxxx/.htpasswduser1:$apr1$/woC1jnP$KAh0SsVn5qeSMjTtn0E9Q0
配置在nginx的配置文件中location /api { auth_basic “Administrator’s Area”; auth_basic_user_file /etc/apache2/.htpasswd; }就OK了.https://www.cnblogs.com/silent2012/p/8377837.html 可以参考
官网reverse proxy的配置在这里samplehttps://nginx.org/en/docs/http/ngx_http_proxy_module.html?_ga=2.27153012.815603832.1587469798-958214196.1587469798#example
############################################################################
docker问题解决, 同下:https://www.cnblogs.com/elson-zeng/p/12553329.htmlCentOS7 firewalld docker 端口映射问题，firewall开放端口后，还是不能访问，解决方案# 宿主机ip: 192.168.91.19docker run -itd --name tomcat -p 8080:8080 tomcat /usr/local/apache-tomcat-9.0.30/bin/startup.sh
# 防火墙放开8080端口firewall-cmd --add-port=8080/tcp --permanent
# 问题：发现访问：192.168.91.19:8080 访问不通，关闭firewall后，又可以访问通了# 解决方案，把docker0网卡添加到trusted域firewall-cmd --permanent --zone=trusted --change-interface=br-d2aa50162455# 重启加载配置firewall-cmd --reload# firewall-cmd相关命令：https://www.cnblogs.com/Raodi/p/11625487.html
我的问题是, 命名防火墙禁用掉了9200端口, 别的机器还是可以访问到.解决方法是,https://docs.docker.com/engine/reference/commandline/dockerd/#daemon-configuration-file#The default location of the configuration file on Linux is /etc/docker/daemon.json. #The --config-file flag can be used to specify a non-default location.docker的daemon.json里配置:"iptables": false --->把iptable禁用掉,--->直接导致,kibana无法访问elasticsearch了然后再按上面博客, 把docker的一个区域加入了trusted里. --->kibana可以访问elasticsearch了.>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>看来碰到问题的人好多#跟我差不多https://www.jianshu.com/p/69d3ab177655https://www.jianshu.com/p/10c467600ef9
#docker service的配置文件在/usr/lib/systemd/system/docker.service#当时配置docker服务的代理 在这里 /etc/systemd/system/docker.service.d/http-proxy.conf
这里涉及到了3个配置文件:/etc/docker/daemon.json #守护进程各种配置, 配置项非常多, 包括docker镜像站点等/usr/lib/systemd/system/docker.service #服务配置文件/etc/systemd/system/docker.service.d/http-proxy.conf #配置代理
所以, 需要学习很多相关的知识, 算是还债. 因为一直没有搞得非常通透, 导致解决问题磕磕绊绊1.systemctl命令相关, 启动脚本在哪里? 配置文件在哪里2.systemctl daemon相关, 配置文件在哪里, 怎么配的
https://www.cnblogs.com/brucewhite/p/11608853.htmlsystemctl --help 查看帮助!!!!!!!systemctl enable | disable | is-enabled | status | is-active unitsystemctl get-default|set-default graphical.target|multi-user.target islate 在线切换模式 systemctl reload-daemon 加载新的unit 配置文件 systemd的unit 的配置文件位置:redhat系列： 目录:/usr/lib/systemd/system/ 下 enable 是在 /etc/systemd/system/multi-user.target.wants/ 这个目录下 做 unit 配置文件的软链：
3.linux自动启动的几种方式, init.d, chkconfig, service等等几个概念https://www.cnblogs.com/liuxia912/p/10960610.html#CentOS 7的服务systemctl脚本存放在：/usr/lib/systemd/，有系统（system）和用户（user）之分，#像需要开机不登陆就能运行的程序，还是存在系统服务里吧，即：/usr/lib/systemd/system目录下#对于那些支持 Systemd 的软件，安装的时候，会自动在/usr/lib/systemd/system目录添加一个配置文件。
在配置vnc服务器的时候也碰到了这个systemctl daemon-reload
>>待续
4.防火墙的几种配置, firewalld和iptables
>>>>>>>>>>>>>>>>>>>>>>>>>>>$ firewall-cmd --zone=public --remove-port=10050/tcp##这一句 runtime-to_permanent$ firewall-cmd --runtime-to-permanent$ firewall-cmd --reload >>>>>>>>>>>>>>>>>>>>>>>>>>>
https://www.tecmint.com/start-stop-disable-enable-firewalld-iptables-firewall/下面几个课题有空再看Understanding IPtables Firewall Basics and TipsConfigure Iptables Firewall in LinuxConfigure FirewallD in LinuxUseful FirewallD Rules to Manage Firewall in LinuxHow to Control Network Traffic Using FirewallD and Iptables
# Also, you can mask the firewall service # which creates a symbolic link of the firewall.service to /dev/null, # thus disabling the service.# This is reverse of masking the service. # This removes the symlink of the service created during masking, # thus re-enabling the service.systemctl statrt/stop/restart/disable/enable/mask/unmask/status firewalldfirewall-cmd --state
systemctl restart firewalld.service(为什么要加一个.service, 是什么意思?)systemctl is-enabled firewalld.service #查看服务是否开机启动systemctl list-unit-files|grep enabled #查看已启动的服务列表systemctl --failed #查看启动失败的服务列表
关于iptables#How to Start/Stop and Enable/Disable IPtables Servicesystemctl start/stop/restart/disable/enable/status iptablesservice iptables start/stop/save/status
#ubuntu/debian?sudo ufw enable/disable/status
iptables -L -n -v
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>关于firewalld, 这个老外写得太好了!https://www.liquidweb.com/kb/an-introduction-to-firewalld/
firewall-cmd --permanent --add-port=22/TCPfirewall-cmd --permanent --add-port=53/UDPfirewall-cmd --permanent --remove-port=444/tcpfirewall-cmd --permanent --add-service=sshfirewall-cmd --permanent --add-service=httpfirewall-cmd --permanent --remove-service=mysql
#whitelist an ip addressfirewall-cmd --permanent --add-source=192.168.1.100#以CIDR地址形式加一个范围, 这是最近学到的.firewall-cmd --permanent --add-source=192.168.1.0/24firewall-cmd --permanent --remove-source=192.168.1.100
Block an IP Address#As the firewall-cmd tool is mostly used for opening or allowing access, #rich rules are needed to block an IP. #Rich rules are similar in form to the way iptables rules are written.firewall-cmd --permanent --add-rich-rule="rule family='ipv4' source address='192.168.1.100' reject"#CIDR rangefirewall-cmd --permanent --add-rich-rule="rule family='ipv4' source address='192.168.1.0/24' reject"
Whitelist an IP Address for a Specific Port (More Rich Rules)firewall-cmd --permanent --add-rich-rule='rule family="ipv4" source address="192.168.1.100" port protocol="tcp" port="3306" accept'firewall-cmd --permanent --remove-rich-rule='rule family="ipv4" source address="192.168.1.100" port protocol="tcp" port="3306" accept'
#savefirewall-cmd --reloadfirewall-cmd --runtime-to-permanent#showfirewall-cmd --list-all
Bonus #1 :: Firewalld GUI Configuration You can find these under Applications > Sundry menu on CentOS 7.x and RedHat 7.x servers that have graphical access to. To install this application, from the command line, you can run the command:sudo yum install firewall-config
Bonus #2 :: Firewalld Direct Rules/etc/firewalld/direct.xml
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>man 1 systemctlSYSTEMCTL(1) systemctl SYSTEMCTL(1)
NAME systemctl - Control the systemd system and service manager
SYNOPSIS systemctl [OPTIONS...] COMMAND [NAME...]
DESCRIPTION systemctl may be used to introspect and control the state of the "systemd" system and service manager. Please refer to systemd(1) for an introduction into the basic concepts and functionality this tool manages.
>>>加固Linuxhttps://www.liquidweb.com/kb/security-for-your-linux-server/
>>>>>中文的在这里, 比较冗长, 没有好好排版, 内容不错.https://www.linuxidc.com/Linux/2018-01/150072.htmfirewall daemon 独立于 system-config-firewall，但二者不能同时使用。如果你想使用自己的 iptables 和 ip6tables 静态防火墙规则, 那么请安装 iptables-services 并且禁用 firewalld ，启用 iptables 和ip6tables:yum install iptables-servicessystemctl mask firewalld.servicesystemctl enable iptables.servicesystemctl enable ip6tables.service
静态防火墙规则配置文件是 /etc/sysconfig/iptables 以及 /etc/sysconfig/ip6tables .
什么是区域？网络区域定义了网络连接的可信等级。这是一个一对多的关系，这意味着一次连接可以仅仅是一个区域的一部分，而一个区域可以用于很多连接。
>>>https://www.cnblogs.com/Raodi/p/11625487.html#_label1
>>>>>>>>>>>>>>>>>>>>>>>>若 Nginx 收到请求为 https://ngxin_server_name/hello/world而 Nginx 代理的路径为 /hello/（即在 location /hello/ 内设置代理）则不以 / 结尾的被代理服务器收到的请求路径是 /hello/world以 / 结尾的被代理服务器收到的请求路径是 /world如果是为了在同一个域名下以不同路径分配不同的APP应选择后者以 / 结尾原文链接：https://blog.csdn.net/randomparty/java/article/details/80961189