hdfs、yarn、hbase这些组件的master支持多个,实现自动主备切换,其中hdfs、hbase无论访问主master或者备master都可以正常访问页面,但是yarn比较特别,只有主master的页面可以访问,备master会返回Refresh,3s后重定向;
一种方式是提供两个域名,分别对应两个yarn的master,一旦有master切换,需要手工切换到另外一个,有没有更好的方式?
访问备master过程如下:
curl http://standby_ip:8088/cluster -v
* About to connect() to standby_ip port 8088 (#0)
* Trying standby_ip...
* Connected to standby_ip (standby_ip) port 8088 (#0)
> GET /cluster HTTP/1.1
> User-Agent: curl/7.29.0
> Host: standby_ip:8088
> Accept: */*
>
< HTTP/1.1 200 OK
< Cache-Control: no-cache
< Expires: Tue, 25 Sep 2018 03:59:22 GMT
< Date: Tue, 25 Sep 2018 03:59:22 GMT
< Pragma: no-cache
< Expires: Tue, 25 Sep 2018 03:59:22 GMT
< Date: Tue, 25 Sep 2018 03:59:22 GMT
< Pragma: no-cache
< Content-Type: text/plain; charset=UTF-8
< Refresh: 3; url=http://active_ip:8088/cluster
< Content-Length: 103
< Server: Jetty(6.1.26)
<
This is standby RM. Redirecting to the current active RM: http://active_ip:8088/cluster
* Connection #0 to host standby_ip left intact
可见备master响应http status为200,包含Refresh头,同时body为This is standby RM. Redirecting to the current active RM:***
有没有可能在load balancer(比如nginx)上配置实现自动切换?这里非法响应需要判断header包含‘Refresh’或者判断body包含‘This is standby RM’
1)被动切换,支持http status以及timeout等判断,不满足
http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_next_upstream
proxy_next_upstream error | timeout | invalid_header | http_500 | http_502 | http_503 | http_504 | http_403 | http_404 | http_429 | non_idempotent | off ...;
2)主动健康检查,最常用的开源module,只支持http status级别的健康检查,不满足
https://github.com/yaoweibin/nginx_upstream_check_module
check_http_expect_alive
syntax: *check_http_expect_alive [ http_2xx | http_3xx | http_4xx |
http_5xx ]*
default: *http_2xx | http_3xx*
context: *upstream*
description: These status codes indicate the upstream server's http
response is ok, the backend is alive.
3)nginx收费版本的ngx_http_upstream_hc_module,支持header和body的判断,可以满足
Dynamically configurable group with periodic health checks is available as part of our commercial subscription:
http://nginx.org/en/docs/http/ngx_http_upstream_hc_module.html
match active {
body !~ "This is standby RM";
}