zoukankan      html  css  js  c++  java
  • fastdfs的入门到精通(fastdfs集群的搭建)

    引言:

      前面已经结束fastdfs的一些理论和单机的相关知识,本篇博客主要讨论fastdfs的集群的搭建。因为在第一篇就介绍了fastdfs的负载和高效,这些都是依靠集群来实现的。

    集群搭建:

      准备工作:

         1. 先准备5台机器

     
    fastdfs集群搭建服务器分布
    tracker1 192.168.236.134
    tracker2 192.168.236.133
    storage1-group1 192.168.236.132
    storage2-group1   192.168.236.135
    storage1-group2 192.168.236.131

        为了搭建效果,我准备5台全新的机器,不在原先演示的fastdfs的虚拟机上进行演示

       2> 关闭每台机器防火墙:    

          启动: systemctl start firewalld
           关闭: systemctl stop firewalld
           查看状态: systemctl status firewalld 
           开机禁用  : systemctl disable firewalld
           开机启用  : systemctl enable firewalld

       3> 上传所需要的安装包: 

          关于下载地址可以参考我起前面几篇博客,也可以去我csdh去下载:https://download.csdn.net/download/xiaohu_0602/12258263  主要包含

          

       到这,准备工作基本完成。下面来看一下搭建步骤。

      第一步:

        克隆五台主机,在关机状态下删除mac地址,或者登进去删除mac地址,效果如下:相关ip地址如上:

         

        执行安装前的准备驱动安装:五台机器都要执行(CRT可以同时执行,后续介绍

          yum install -y gcc-c++ perl-devel pcre-devel openssl-devel zlib-devel wget

       第二步: 解压libcommon和fastdfs   然后进行编译

        连接到tracker1 服务器上进行以下操作:

        进入到fastdfs文件夹下:在目录下解压libcommon和fastdfs安装包: 

           --注意: 其实集群化搭建和单机搭建步骤相同,我再操作一遍,增加大家印象,建议大家自己尝试搭建一下

        

         把解压好的fastdfs和lib两个文件夹使用scp拷贝到其他主机,或者,你去每台主机都去解压一次

       tracker1的主机ip是192.168.236.132     

        scp -r fastdfs/ 192.168.236.133:/root -- 输入密码确认即可 scp -r fastdfs/ 192.168.236.132:/root -- 输入密码确认即可 scp -r fastdfs/ 192.168.236.135:/root -- 输入密码确认即可 scp -r fastdfs/ 192.168.236.131:/root -- 输入密码确认即可

      大家注意: 我们使用的连接工具的不同也会有不同的操作,如果是xshell 只能上传文件,不能上传文件夹,可以传压缩包。如果使用secureCRT的话,我们可以直接上传一个文件夹,也可以操作所有的连接。后续我会接受CRT的操作,此篇博客暂时不做重点

      解压后先进入 libfastcommon-1.0.36    执行./make.sh   && ./make.sh install  

      然后再进入:fastdfs-5.11   执行./make.sh   && ./make.sh install    

      截图我就不贴出来了。大家按照顺序执行就行了。

      在上面执行完成后,会在/etc/fdfs/   会生成相应文件

    第三步: 

        配置tracker服务器:

        先配置tracker1  (192.168.236.132这台。另外一台吧tracker.conf   copy过去就行)  

          cd /etc/fdfs/
          mv tracker.conf.sample tracker.conf
          mv /etc/fdfs/tracker.conf.sample /etc/fdfs/tracker.conf     --两台tracker机器同时修改
          mkdir -p /data/tracker   -- 创建保存
          vim tracker.conf
            base_path=/data/tracker -- 保存目录修改

        tracker2 服务器操作相同

       启动两台tracker服务 

          fdfs_trackerd /etc/fdfs/tracker.conf  start  -- 在两台tracker服务上执行  

       可以查询一下fdfs是否启动:

          ps aux | grep fdfs  (ps -ef | grep fdfs)  

       第四步:配置storage服务器:

        提示: 大家在看我大家集群中,会发现我准备了3台服务器,一台  group1 有两台。。group2 有一台,顺便把负载也搭建起来,仅供参考

      关于storage的配置也相同,,只需要把tracker服务器的地址配置上,然后修改相关的group=xx就行了。下面我来操作一下:

          1>解压libcommon tar -zxvf libfastcommon-1.0.36.tar.gz 
          2> 编译:
            cd libfastcommon-1.0.36 
            ./make.sh && ./make.sh install
          3> 解压fastdfs tar -zxvf fastdfs-5.11.tar.gz 
          4> 编译: ./make.sh && ./make.sh install

            ---- 这几步和上面配置tracker服务器一样

          5> 修改storage.conf    

          cd /etc/fdfs/
          mv storage.conf.sample storage.conf
          mv /etc/fdfs/storage.conf.sample /etc/fdfs/storage.conf     --3台storage机器同时修改
          mkdir -p /data/storage   -- 创建保存
          vim /etc/fdfs/storage.conf
          group = group1 -- 注意其中两台是默认不用修改的,那台group2的主机要把这个修改group2 base_path
    =/data/storage store_path0=/data/storage tracker_server=192.168.236.133:22122 --下面是配置多storage的地址即可 tracker_server=192.168.236.134:22122

          6> 修改后启动storage服务

          7> 启动没问题后,吧storage.conf  同步到另外两台机器

           scp -r /etc/fdfs/storage.conf  192.168.236.135:/etc/fdfs/   -- 这台是storage2-group1   配置文件不用修改
                scp -r /etc/fdfs/storage.conf  192.168.236.131:/etc/fdfs/   -- 这台是storage1-group2   配置文件需要修改组名为group2

          8> 启动另外两台storage服务:

            fdfs_storaged /etc/fdfs/storage.conf start

      第五步:集群验证:

        上面我们已经把tracker服务和storage服务都启动了,那怎么知道对不对呢。下面我们验证一下: 使用 fdfs_moniter来验证

          mv /etc/fdfs/client.conf.sample /etc/fdfs/client.conf
             mkdir -p /data/client
             vi client.conf    
                base_path=/data/client
                tracker_server=192.168.236.133:22122
                tracker_server=192.168.236.134:22122

        执行: fdfs_monitor /etc/fdfs/client.conf   效果如下:  重点看我标红的部分

    [root@localhost ~]# fdfs_monitor /etc/fdfs/client.conf 
    [2020-03-19 12:59:52] DEBUG - base_path=/data/client, connect_timeout=30, network_timeout=60, tracker_server_count=2, anti_steal_token=0, anti_steal_secret_key length=0, use_connection_pool=0, g_connection_pool_max_idle_time=3600s, use_storage_id=0, storage server id count: 0
    
    server_count=2, server_index=1
    
    tracker server is 192.168.236.134:22122
    
    group count: 2
    
    Group 1:
    group name = group1
    disk total space = 17394 MB
    disk free space = 15638 MB
    trunk free space = 0 MB
    storage server count = 2
    active server count = 2
    storage server port = 23000
    storage HTTP port = 8888
    store path count = 1
    subdir count per path = 256
    current write server index = 0
    current trunk file id = 0
    
            Storage 1:
                    id = 192.168.236.132
                    ip_addr = 192.168.236.132 (localhost.localdomain)  ACTIVE
                    http domain = 
                    version = 5.11
                    join time = 2020-03-19 12:42:22
                    up time = 2020-03-19 12:42:22
                    total storage = 17394 MB
                    free storage = 15641 MB
                    upload priority = 10
                    store_path_count = 1
                    subdir_count_per_path = 256
                    storage_port = 23000
                    storage_http_port = 8888
                    current_write_path = 0
                    source storage id = 
                    if_trunk_server = 0
                    connection.alloc_count = 256
                    connection.current_count = 1
                    connection.max_count = 1
                    total_upload_count = 0
                    success_upload_count = 0
                    total_append_count = 0
                    success_append_count = 0
                    total_modify_count = 0
                    success_modify_count = 0
                    total_truncate_count = 0
                    success_truncate_count = 0
                    total_set_meta_count = 0
                    success_set_meta_count = 0
                    total_delete_count = 0
                    success_delete_count = 0
                    total_download_count = 0
                    success_download_count = 0
                    total_get_meta_count = 0
                    success_get_meta_count = 0
                    total_create_link_count = 0
                    success_create_link_count = 0
                    total_delete_link_count = 0
                    success_delete_link_count = 0
                    total_upload_bytes = 0
                    success_upload_bytes = 0
                    total_append_bytes = 0
                    success_append_bytes = 0
                    total_modify_bytes = 0
                    success_modify_bytes = 0
                    stotal_download_bytes = 0
                    success_download_bytes = 0
                    total_sync_in_bytes = 0
                    success_sync_in_bytes = 0
                    total_sync_out_bytes = 0
                    success_sync_out_bytes = 0
                    total_file_open_count = 0
                    success_file_open_count = 0
                    total_file_read_count = 0
                    success_file_read_count = 0
                    total_file_write_count = 0
                    success_file_write_count = 0
                    last_heart_beat_time = 2020-03-19 12:59:33
                    last_source_update = 1969-12-31 19:00:00
                    last_sync_update = 1969-12-31 19:00:00
                    last_synced_timestamp = 1969-12-31 19:00:00 
            Storage 2:
                    id = 192.168.236.135
                    ip_addr = 192.168.236.135  ACTIVE
                    http domain = 
                    version = 5.11
                    join time = 2020-03-19 12:48:52
                    up time = 2020-03-19 12:48:52
                    total storage = 17394 MB
                    free storage = 15638 MB
                    upload priority = 10
                    store_path_count = 1
                    subdir_count_per_path = 256
                    storage_port = 23000
                    storage_http_port = 8888
                    current_write_path = 0
                    source storage id = 192.168.236.132
                    if_trunk_server = 0
                    connection.alloc_count = 256
                    connection.current_count = 1
                    connection.max_count = 1
                    total_upload_count = 0
                    success_upload_count = 0
                    total_append_count = 0
                    success_append_count = 0
                    total_modify_count = 0
                    success_modify_count = 0
                    total_truncate_count = 0
                    success_truncate_count = 0
                    total_set_meta_count = 0
                    success_set_meta_count = 0
                    total_delete_count = 0
                    success_delete_count = 0
                    total_download_count = 0
                    success_download_count = 0
                    total_get_meta_count = 0
                    success_get_meta_count = 0
                    total_create_link_count = 0
                    success_create_link_count = 0
                    total_delete_link_count = 0
                    success_delete_link_count = 0
                    total_upload_bytes = 0
                    success_upload_bytes = 0
                    total_append_bytes = 0
                    success_append_bytes = 0
                    total_modify_bytes = 0
                    success_modify_bytes = 0
                    stotal_download_bytes = 0
                    success_download_bytes = 0
                    total_sync_in_bytes = 0
                    success_sync_in_bytes = 0
                    total_sync_out_bytes = 0
                    success_sync_out_bytes = 0
                    total_file_open_count = 0
                    success_file_open_count = 0
                    total_file_read_count = 0
                    success_file_read_count = 0
                    total_file_write_count = 0
                    success_file_write_count = 0
                    last_heart_beat_time = 2020-03-19 12:59:19
                    last_source_update = 1969-12-31 19:00:00
                    last_sync_update = 1969-12-31 19:00:00
                    last_synced_timestamp = 1969-12-31 19:00:00 
    
    Group 2:
    group name = group2
    disk total space = 17394 MB
    disk free space = 15638 MB
    trunk free space = 0 MB
    storage server count = 1
    active server count = 1
    storage server port = 23000
    storage HTTP port = 8888
    store path count = 1
    subdir count per path = 256
    current write server index = 0
    current trunk file id = 0
    
            Storage 1:
                    id = 192.168.236.131
                    ip_addr = 192.168.236.131  ACTIVE
                    http domain = 
                    version = 5.11
                    join time = 2020-03-19 12:48:45
                    up time = 2020-03-19 12:48:45
                    total storage = 17394 MB
                    free storage = 15638 MB
                    upload priority = 10
                    store_path_count = 1
                    subdir_count_per_path = 256
                    storage_port = 23000
                    storage_http_port = 8888
                    current_write_path = 0
                    source storage id = 
                    if_trunk_server = 0
                    connection.alloc_count = 256
                    connection.current_count = 0
                    connection.max_count = 0
                    total_upload_count = 0
                    success_upload_count = 0
                    total_append_count = 0
                    success_append_count = 0
                    total_modify_count = 0
                    success_modify_count = 0
                    total_truncate_count = 0
                    success_truncate_count = 0
                    total_set_meta_count = 0
                    success_set_meta_count = 0
                    total_delete_count = 0
                    success_delete_count = 0
                    total_download_count = 0
                    success_download_count = 0
                    total_get_meta_count = 0
                    success_get_meta_count = 0
                    total_create_link_count = 0
                    success_create_link_count = 0
                    total_delete_link_count = 0
                    success_delete_link_count = 0
                    total_upload_bytes = 0
                    success_upload_bytes = 0
                    total_append_bytes = 0
                    success_append_bytes = 0
                    total_modify_bytes = 0
                    success_modify_bytes = 0
                    stotal_download_bytes = 0
                    success_download_bytes = 0
                    total_sync_in_bytes = 0
                    success_sync_in_bytes = 0
                    total_sync_out_bytes = 0
                    success_sync_out_bytes = 0
                    total_file_open_count = 0
                    success_file_open_count = 0
                    total_file_read_count = 0
                    success_file_read_count = 0
                    total_file_write_count = 0
                    success_file_write_count = 0
                    last_heart_beat_time = 2020-03-19 12:59:47
                    last_source_update = 1969-12-31 19:00:00
                    last_sync_update = 1969-12-31 19:00:00
                    last_synced_timestamp = 1969-12-31 19:00:00 
    [root@localhost ~]# 

      到这集群就搭建完成了,有不明白的或者我写错的可以留言


    补充: 

      大家注意: 上面集群只是默认大家完成,关于很多配置需要在tracker.conf进行适当调整,本篇只做集群搭建的讲解,暂不深入讨论配置文件  

  • 相关阅读:
    Swagger接入
    Elasticsearch5.0.1索引压测结果
    Elasticsearch5.0 BreakChange摘要
    Elasticsearch1.7到2.3升级实践总结
    Java动态代理全面分析
    Spring之AntPathMatcher
    Lock的实现之ReentrantLock详解
    Elasticsearch之client源码简要分析
    elasticserach 索引删除 源码分析
    httpClient4.5.2工具类总结
  • 原文地址:https://www.cnblogs.com/huhongy/p/12531236.html
Copyright © 2011-2022 走看看