引言:
前面已经结束fastdfs的一些理论和单机的相关知识,本篇博客主要讨论fastdfs的集群的搭建。因为在第一篇就介绍了fastdfs的负载和高效,这些都是依靠集群来实现的。
集群搭建:
准备工作:
1. 先准备5台机器
fastdfs集群搭建服务器分布 | |
tracker1 | 192.168.236.134 |
tracker2 | 192.168.236.133 |
storage1-group1 | 192.168.236.132 |
storage2-group1 | 192.168.236.135 |
storage1-group2 | 192.168.236.131 |
为了搭建效果,我准备5台全新的机器,不在原先演示的fastdfs的虚拟机上进行演示
2> 关闭每台机器防火墙:
启动: systemctl start firewalld
关闭: systemctl stop firewalld
查看状态: systemctl status firewalld
开机禁用 : systemctl disable firewalld
开机启用 : systemctl enable firewalld
3> 上传所需要的安装包:
关于下载地址可以参考我起前面几篇博客,也可以去我csdh去下载:https://download.csdn.net/download/xiaohu_0602/12258263 主要包含
到这,准备工作基本完成。下面来看一下搭建步骤。
第一步:
克隆五台主机,在关机状态下删除mac地址,或者登进去删除mac地址,效果如下:相关ip地址如上:
执行安装前的准备驱动安装:五台机器都要执行(CRT可以同时执行,后续介绍)
yum install -y gcc-c++ perl-devel pcre-devel openssl-devel zlib-devel wget
第二步: 解压libcommon和fastdfs 然后进行编译
连接到tracker1 服务器上进行以下操作:
进入到fastdfs文件夹下:在目录下解压libcommon和fastdfs安装包:
--注意: 其实集群化搭建和单机搭建步骤相同,我再操作一遍,增加大家印象,建议大家自己尝试搭建一下
把解压好的fastdfs和lib两个文件夹使用scp拷贝到其他主机,或者,你去每台主机都去解压一次
tracker1的主机ip是192.168.236.132
scp -r fastdfs/ 192.168.236.133:/root -- 输入密码确认即可 scp -r fastdfs/ 192.168.236.132:/root -- 输入密码确认即可 scp -r fastdfs/ 192.168.236.135:/root -- 输入密码确认即可 scp -r fastdfs/ 192.168.236.131:/root -- 输入密码确认即可
大家注意: 我们使用的连接工具的不同也会有不同的操作,如果是xshell 只能上传文件,不能上传文件夹,可以传压缩包。如果使用secureCRT的话,我们可以直接上传一个文件夹,也可以操作所有的连接。后续我会接受CRT的操作,此篇博客暂时不做重点
解压后先进入 libfastcommon-1.0.36 执行./make.sh && ./make.sh install
然后再进入:fastdfs-5.11 执行./make.sh && ./make.sh install
截图我就不贴出来了。大家按照顺序执行就行了。
在上面执行完成后,会在/etc/fdfs/ 会生成相应文件
第三步:
配置tracker服务器:
先配置tracker1 (192.168.236.132这台。另外一台吧tracker.conf copy过去就行)
cd /etc/fdfs/ mv tracker.conf.sample tracker.conf mv /etc/fdfs/tracker.conf.sample /etc/fdfs/tracker.conf --两台tracker机器同时修改 mkdir -p /data/tracker -- 创建保存 vim tracker.conf
base_path=/data/tracker -- 保存目录修改
tracker2 服务器操作相同
启动两台tracker服务
fdfs_trackerd /etc/fdfs/tracker.conf start -- 在两台tracker服务上执行
可以查询一下fdfs是否启动:
ps aux | grep fdfs (ps -ef | grep fdfs)
第四步:配置storage服务器:
提示: 大家在看我大家集群中,会发现我准备了3台服务器,一台 group1 有两台。。group2 有一台,顺便把负载也搭建起来,仅供参考
关于storage的配置也相同,,只需要把tracker服务器的地址配置上,然后修改相关的group=xx就行了。下面我来操作一下:
1>解压libcommon tar -zxvf libfastcommon-1.0.36.tar.gz
2> 编译:
cd libfastcommon-1.0.36
./make.sh && ./make.sh install
3> 解压fastdfs tar -zxvf fastdfs-5.11.tar.gz
4> 编译: ./make.sh && ./make.sh install
---- 这几步和上面配置tracker服务器一样
5> 修改storage.conf
cd /etc/fdfs/
mv storage.conf.sample storage.conf
mv /etc/fdfs/storage.conf.sample /etc/fdfs/storage.conf --3台storage机器同时修改
mkdir -p /data/storage -- 创建保存
vim /etc/fdfs/storage.conf
group = group1 -- 注意其中两台是默认不用修改的,那台group2的主机要把这个修改group2 base_path=/data/storage store_path0=/data/storage tracker_server=192.168.236.133:22122 --下面是配置多storage的地址即可 tracker_server=192.168.236.134:22122
6> 修改后启动storage服务
7> 启动没问题后,吧storage.conf 同步到另外两台机器
scp -r /etc/fdfs/storage.conf 192.168.236.135:/etc/fdfs/ -- 这台是storage2-group1 配置文件不用修改 scp -r /etc/fdfs/storage.conf 192.168.236.131:/etc/fdfs/ -- 这台是storage1-group2 配置文件需要修改组名为group2
8> 启动另外两台storage服务:
fdfs_storaged /etc/fdfs/storage.conf start
第五步:集群验证:
上面我们已经把tracker服务和storage服务都启动了,那怎么知道对不对呢。下面我们验证一下: 使用 fdfs_moniter来验证
mv /etc/fdfs/client.conf.sample /etc/fdfs/client.conf mkdir -p /data/client vi client.conf base_path=/data/client tracker_server=192.168.236.133:22122 tracker_server=192.168.236.134:22122
执行: fdfs_monitor /etc/fdfs/client.conf 效果如下: 重点看我标红的部分
[root@localhost ~]# fdfs_monitor /etc/fdfs/client.conf [2020-03-19 12:59:52] DEBUG - base_path=/data/client, connect_timeout=30, network_timeout=60, tracker_server_count=2, anti_steal_token=0, anti_steal_secret_key length=0, use_connection_pool=0, g_connection_pool_max_idle_time=3600s, use_storage_id=0, storage server id count: 0 server_count=2, server_index=1 tracker server is 192.168.236.134:22122 group count: 2 Group 1: group name = group1 disk total space = 17394 MB disk free space = 15638 MB trunk free space = 0 MB storage server count = 2 active server count = 2 storage server port = 23000 storage HTTP port = 8888 store path count = 1 subdir count per path = 256 current write server index = 0 current trunk file id = 0 Storage 1: id = 192.168.236.132 ip_addr = 192.168.236.132 (localhost.localdomain) ACTIVE http domain = version = 5.11 join time = 2020-03-19 12:42:22 up time = 2020-03-19 12:42:22 total storage = 17394 MB free storage = 15641 MB upload priority = 10 store_path_count = 1 subdir_count_per_path = 256 storage_port = 23000 storage_http_port = 8888 current_write_path = 0 source storage id = if_trunk_server = 0 connection.alloc_count = 256 connection.current_count = 1 connection.max_count = 1 total_upload_count = 0 success_upload_count = 0 total_append_count = 0 success_append_count = 0 total_modify_count = 0 success_modify_count = 0 total_truncate_count = 0 success_truncate_count = 0 total_set_meta_count = 0 success_set_meta_count = 0 total_delete_count = 0 success_delete_count = 0 total_download_count = 0 success_download_count = 0 total_get_meta_count = 0 success_get_meta_count = 0 total_create_link_count = 0 success_create_link_count = 0 total_delete_link_count = 0 success_delete_link_count = 0 total_upload_bytes = 0 success_upload_bytes = 0 total_append_bytes = 0 success_append_bytes = 0 total_modify_bytes = 0 success_modify_bytes = 0 stotal_download_bytes = 0 success_download_bytes = 0 total_sync_in_bytes = 0 success_sync_in_bytes = 0 total_sync_out_bytes = 0 success_sync_out_bytes = 0 total_file_open_count = 0 success_file_open_count = 0 total_file_read_count = 0 success_file_read_count = 0 total_file_write_count = 0 success_file_write_count = 0 last_heart_beat_time = 2020-03-19 12:59:33 last_source_update = 1969-12-31 19:00:00 last_sync_update = 1969-12-31 19:00:00 last_synced_timestamp = 1969-12-31 19:00:00 Storage 2: id = 192.168.236.135 ip_addr = 192.168.236.135 ACTIVE http domain = version = 5.11 join time = 2020-03-19 12:48:52 up time = 2020-03-19 12:48:52 total storage = 17394 MB free storage = 15638 MB upload priority = 10 store_path_count = 1 subdir_count_per_path = 256 storage_port = 23000 storage_http_port = 8888 current_write_path = 0 source storage id = 192.168.236.132 if_trunk_server = 0 connection.alloc_count = 256 connection.current_count = 1 connection.max_count = 1 total_upload_count = 0 success_upload_count = 0 total_append_count = 0 success_append_count = 0 total_modify_count = 0 success_modify_count = 0 total_truncate_count = 0 success_truncate_count = 0 total_set_meta_count = 0 success_set_meta_count = 0 total_delete_count = 0 success_delete_count = 0 total_download_count = 0 success_download_count = 0 total_get_meta_count = 0 success_get_meta_count = 0 total_create_link_count = 0 success_create_link_count = 0 total_delete_link_count = 0 success_delete_link_count = 0 total_upload_bytes = 0 success_upload_bytes = 0 total_append_bytes = 0 success_append_bytes = 0 total_modify_bytes = 0 success_modify_bytes = 0 stotal_download_bytes = 0 success_download_bytes = 0 total_sync_in_bytes = 0 success_sync_in_bytes = 0 total_sync_out_bytes = 0 success_sync_out_bytes = 0 total_file_open_count = 0 success_file_open_count = 0 total_file_read_count = 0 success_file_read_count = 0 total_file_write_count = 0 success_file_write_count = 0 last_heart_beat_time = 2020-03-19 12:59:19 last_source_update = 1969-12-31 19:00:00 last_sync_update = 1969-12-31 19:00:00 last_synced_timestamp = 1969-12-31 19:00:00 Group 2: group name = group2 disk total space = 17394 MB disk free space = 15638 MB trunk free space = 0 MB storage server count = 1 active server count = 1 storage server port = 23000 storage HTTP port = 8888 store path count = 1 subdir count per path = 256 current write server index = 0 current trunk file id = 0 Storage 1: id = 192.168.236.131 ip_addr = 192.168.236.131 ACTIVE http domain = version = 5.11 join time = 2020-03-19 12:48:45 up time = 2020-03-19 12:48:45 total storage = 17394 MB free storage = 15638 MB upload priority = 10 store_path_count = 1 subdir_count_per_path = 256 storage_port = 23000 storage_http_port = 8888 current_write_path = 0 source storage id = if_trunk_server = 0 connection.alloc_count = 256 connection.current_count = 0 connection.max_count = 0 total_upload_count = 0 success_upload_count = 0 total_append_count = 0 success_append_count = 0 total_modify_count = 0 success_modify_count = 0 total_truncate_count = 0 success_truncate_count = 0 total_set_meta_count = 0 success_set_meta_count = 0 total_delete_count = 0 success_delete_count = 0 total_download_count = 0 success_download_count = 0 total_get_meta_count = 0 success_get_meta_count = 0 total_create_link_count = 0 success_create_link_count = 0 total_delete_link_count = 0 success_delete_link_count = 0 total_upload_bytes = 0 success_upload_bytes = 0 total_append_bytes = 0 success_append_bytes = 0 total_modify_bytes = 0 success_modify_bytes = 0 stotal_download_bytes = 0 success_download_bytes = 0 total_sync_in_bytes = 0 success_sync_in_bytes = 0 total_sync_out_bytes = 0 success_sync_out_bytes = 0 total_file_open_count = 0 success_file_open_count = 0 total_file_read_count = 0 success_file_read_count = 0 total_file_write_count = 0 success_file_write_count = 0 last_heart_beat_time = 2020-03-19 12:59:47 last_source_update = 1969-12-31 19:00:00 last_sync_update = 1969-12-31 19:00:00 last_synced_timestamp = 1969-12-31 19:00:00 [root@localhost ~]#
到这集群就搭建完成了,有不明白的或者我写错的可以留言
补充:
大家注意: 上面集群只是默认大家完成,关于很多配置需要在tracker.conf进行适当调整,本篇只做集群搭建的讲解,暂不深入讨论配置文件