zoukankan      html  css  js  c++  java

  • CentOS和Ubuntu下安装配置Greenplum数据库集群（包括安装包和源码编译安装）

    http://blog.csdn.net/prettyshuang/article/details/50501904

    首先说一下，无论是CentOS/RedHat还是Ubuntu都可以按源码方式、安装包方式编译安装。

    1.   规划

    192.168.4.93（h93）       1个主master  2个主segment、2个镜像segment

    192.168.4.94（h94）       1个备master  2个主segment、2个镜像segment

    都安装在/home/wslu/gp/gpsql目录。

    2.   安装依赖

    在h93和h94安装以下依赖：

    2.1.  Ubuntu

    [python] view plain copy
     
    1. apt-get install -y git-core  
    2. apt-get install -y gcc g++  
    3. apt-get install -y ccache  
    4. apt-get install -y libreadline-dev  
    5. apt-get install -y bison flex  
    6. apt-get install -y zlib1g-dev  
    7. apt-get install -y openssl libssl-dev  
    8. apt-get install -y libpam-dev  
    9. apt-get install -y libcurl4-dev  
    10. apt-get install -y libbz2-dev  
    11. apt-get install -y python-dev  
    12. apt-get install -y ssh  
    13.    
    14. apt-get install -y libcurl4-dev  
    15. Package libcurl4-dev is a virtual package provided by:  
    16. libcurl4-openssl-dev 7.38.0-4+deb8u2  
    17. libcurl4-nss-dev 7.38.0-4+deb8u2  
    18. libcurl4-gnutls-dev 7.38.0-4+deb8u2  
    19.    
    20. apt-get install -y python-pip  
    21.    
    22. pip install lockfile  
    23. pip install paramiko  
    24. pip install setuptools  
    25. pip install epydoc  
    26. pip install psi  
    27. Note: debian8 required pip install --pre psi  

    2.2.  Centos

    yum源要按照我的总结中的设置。

    [python] view plain copy
     
    1. yum install –y git.x86_64  
    2. yum install –y gcc.x86_64 gcc-c++.x86_64  
    3. yum install –y ccache.x86_64  
    4. yum install readline.x86_64 readline-devel.x86_64  
    5. yum install bison.x86_64 bison-devel.x86_64  
    6. yum install flex.x86_64 flex-devel.x86_64  
    7. yum install zlib.x86_64 zlib-devel.x86_64  
    8. yum install -y openssl.x86_64 openssl-devel.x86_64  
    9. yum install -y pam.x86_64 pam-devel.x86_64  
    10. yum install –y libcurl.x86_64 libcurl-devel.x86_64  
    11. yum install bzip2-libs.x86_64 bzip2.x86_64 bzip2-devel.x86_64  
    12. yum install libssh2.x86_64 libssh2-devel.x86_64  
    13. yum install python-devel.x86_64  
    14. yum install -y python-pip.noarch  
    15.    
    16. 接着执行：  
    17. pip install lockfile  
    18. pip install paramiko  
    19. pip install setuptools  
    20. pip install epydoc  
    21. pip install psi  
    22. 或者执行：  
    23. yum install python-lockfile.noarch  
    24. yum install python-PSI.x86_64  
    25. yum install python-paramiko.noarch  
    26. yum install python-setuptools.noarch  
    27. yum install epydoc.noarch  

    3.   源码安装

    3.1.  克隆源码

    在h93和h94执行：

    [python] view plain copy
     
    1. $ mkdir /home/wslu/gp/greenplum  
    2. $ cd /home/wslu/gp/greenplum  
    3. $ git clone https://github.com/greenplum-db/gpdb.  

    3.2.  编译安装

    在h93和h94执行：

    [python] view plain copy
     
    1. $ cd /home/wslu/gp/greenplum  
    2. $ CFLAGS+="-O2" ./configure--prefix=/home/wslu/gp/gpsql --enable-debug --enable-depend --enable-cassert  
    3. $ make  
    4. $ make install  

    安装时如果遇到某些python包(lockfile、 paramiko、PSI等)找不到，可以参考HAWQ， (源码https://github.com/apache/incubator-hawq)<hawq_src>/tools/bin/pythonSrc/ 下所有的压缩包拷贝到/home/wslu/gp/greenplum/gpMgmt/bin/pythonSrc/ext/ 中，然后再make install即可。

    至此集群源码编译完成。

    4.   安装包安装

    下载greenplum-db-4.3.6.1-build-2-RHEL5-x86_64.zip

    [python] view plain copy
     
    1. $ unzip greenplum-db-4.3.6.1-build-2-RHEL5-x86_64.zip  

    在h93和h94使用普通用户安装：

    [python] view plain copy
     
    1. $./greenplum-db-4.3.6.1-build-2-RHEL5-x86_64.bin   

    安装到 /home/wslu/gp/gpsql

    5.   设置操作系统参数

    在h93和h94执行下述设置：

    5.1.  系统级参数配置

    以下配置需要在root下进行配置，所以需要能够得到root权限。

    首先是防火墙的设置：

    [python] view plain copy
     
    1. service iptables status    #查看是否开启了防火墙  
    2. service iptables stop       #关闭防火墙  
    3. chkconfig iptables off      #设置开机不会启动防火墙  

    其次是ssh的速度设置：

    [python] view plain copy
     
    1. sed -i 's/^GSS/#&/g' /etc/ssh/sshd_config      #用来加速SSH连接的  
    2.   service sshd reload  

    内核处理和内存方面的设置：

    [python] view plain copy
     
    1. # 设置内核参数, 并在启动时生效  
    2. sysctl -p - >>/etc/sysctl.conf <<EOF  
    3. # configurations  
    4. kernel.sysrq=1  
    5. kernel.core_pattern=core  
    6. kernel.core_uses_pid=1  
    7. kernel.msgmnb=65536  
    8. kernel.msgmax=65536  
    9. kernel.msgmni=2048  
    10. kernel.sem=25600 3200000 10000 14200  
    11. net.ipv4.tcp_syncookies=1  
    12. net.ipv4.ip_forward=0  
    13. net.ipv4.conf.default.accept_source_route=0  
    14. net.ipv4.tcp_tw_recycle=1  
    15. net.ipv4.tcp_max_syn_backlog=4096  
    16. net.ipv4.conf.all.arp_filter=1  
    17. net.ipv4.ip_local_port_range=1025 65535  
    18. net.core.netdev_max_backlog=10000  
    19. net.core.rmem_max=2097152  
    20. net.core.wmem_max=2097152  
    21. vm.overcommit_memory=1  
    22. EOF  

    如果配置集群时，因为要设置的共享内存大于系统最大共享内存而报错，则可以按照以下官方推荐的设置配置：

    [python] view plain copy
     
    1. # vi /etc/sysctl.conf  
    2. kernel.shmmax = 500000000  
    3. kernel.shmmni = 4096  
    4. kernel.shmall = 4000000000  
    5. kernel.sem = 250 512000 100 2048  
    6. kernel.sysrq = 1   
    7. kernel.core_uses_pid = 1   
    8. kernel.msgmnb = 65536  
    9. kernel.msgmax = 65536  
    10. kernel.msgmni = 2048  
    11. net.ipv4.tcp_syncookies = 1   
    12. net.ipv4.ip_forward = 0   
    13. net.ipv4.conf.default.accept_source_route = 0   
    14. net.ipv4.tcp_tw_recycle = 1   
    15. net.ipv4.tcp_max_syn_backlog = 4096  
    16. net.ipv4.conf.all.arp_filter = 1   
    17. net.ipv4.ip_local_port_range = 1025 65535  
    18. net.core.netdev_max_backlog = 10000  
    19. net.core.rmem_max = 2097152  
    20. net.core.wmem_max = 2097152  
    21. vm.overcommit_memory = 2  



    文件读写方面的设置：

    [python] view plain copy
     
    1. # 设置limits  
    2. cat >>/etc/security/limits.d/99-kingbasedbcloud.conf <<EOF  
    3. #KingbaseDBCloud configurations  
    4. * soft nofile 65536  
    5. * hard nofile 65536  
    6. * soft nproc 131072  
    7. * hard nproc 131072  
    8. EOF  

    5.2. 数据库级的参数设置

    GUC参数设置，此设置需要按照实际机器来进行配置，常用的配置有如下几点：

    [python] view plain copy
     
    1. work_mem=1GB  
    2. shared_buffers=2GB  
    3. max_connections=500  
    4. max_pool_size=2000  
    5. enable_mergejoin=off  
    6. enable_nestloop=off  
    7. max_prepared_transactions=50  
    8. autovacuum=off  
    9. interconnect_setup_timeout=1200  

    6.   demo集群

    安装完成后，就可以使用demo集群了。

    执行下面命令即可在本机创建包含3个segment，3个segment-mirror，1个master的集群：

    [python] view plain copy
     
    1. $ cd /home/wslu/gp/gpsql  
    2. $ source greenplum_path.sh  
    3. $ gpssh-exkeys –h localhost  
    4. $ cd gpAux/gpdemo  
    5. $ make cluster  
    6. $ source gpdemo-env.sh  

    如果不想用demo集群，可以直接跳过本小节。

    7.   设置环境变量

    [python] view plain copy
     
    1. $ source gpsql/greenplum_path.sh  
    2. $ exportMASTER_DATA_DIRECTORY=/home/wslu/gp/gpsql/data/master/gpseg-1  

    8.   交换key

    在h93执行：

    [python] view plain copy
     
    1. $ gpssh-exkeys –h h94  

             会提示输入h94的wslu用户的密码。（用户必须和h93相同）

    则h93和h94之间可以无密码登陆了。

    9. 初始化集群

    （1）       在h93和h94执行下述指令，以创建数据目录：

    [python] view plain copy
     
    1. $ mkdir gpsql/data/primary gpsql/data/mirror gpsql/data/master –p  

    （2）       在h93创建配置文件configs/gpinitsystem_config，内容如下：

    [python] view plain copy
     
    1. ARRAY_NAME="EMC Greenplum DW"  
    2. SEG_PREFIX=gpseg  
    3. PORT_BASE=40000  
    4. declare -a DATA_DIRECTORY=(/home/wslu/gp/gpsql/data/primary /home/wslu/gp/gpsql/data/primary)  
    5. MASTER_HOSTNAME=h93  
    6. MASTER_DIRECTORY=/home/wslu/gp/gpsql/data/master  
    7. MASTER_PORT=5432  
    8. TRUSTED_SHELL=ssh  
    9. CHECK_POINT_SEGMENTS=8  
    10. ENCODING=UNICODE  
    11. MIRROR_PORT_BASE=50000  
    12. REPLICATION_PORT_BASE=41000  
    13. MIRROR_REPLICATION_PORT_BASE=51000  
    14. declare -a MIRROR_DATA_DIRECTORY=(/home/wslu/gp/gpsql/data/mirror /home/wslu/gp/gpsql/data/mirror)  

    注意：configs目录是我自己创建的、便于保存自定义配置文件的目录。该步骤的目的是创建一个初始化时要用的配置文件，并没有路径的要求。

    （3）       在h93创建配置文件configs/hostfile_gpinitsystem，内容如下：

    [python] view plain copy
     
    1. h93  
    2. h94  

    注意：configs目录是我自己创建的、便于保存自定义配置文件的目录。该步骤的目的是创建一个初始化时要用的配置文件，并没有路径的要求。

    （4）       在h93执行下述指令初始化集群：

    [python] view plain copy
     
    1. [wslu@h93 gpsql]$ gpinitsystem -c configs/gpinitsystem_config -h configs/hostfile_gpinitsystem –a  
    2. 20160114:14:30:03:005980 gpinitsystem:h93:wslu-[INFO]:-Checking configuration parameters, please wait...  
    3. 20160114:14:30:03:005980 gpinitsystem:h93:wslu-[INFO]:-Reading Greenplum configuration file configs/gpinitsystem_config  
    4. 20160114:14:30:03:005980 gpinitsystem:h93:wslu-[INFO]:-Locale has not been set in configs/gpinitsystem_config, will set to default value  
    5. 20160114:14:30:03:005980 gpinitsystem:h93:wslu-[INFO]:-Locale set to en_US.utf8  
    6. 20160114:14:30:03:005980 gpinitsystem:h93:wslu-[INFO]:-No DATABASE_NAME set, will exit following template1 updates  
    7. 20160114:14:30:03:005980 gpinitsystem:h93:wslu-[INFO]:-MASTER_MAX_CONNECT not set, will set to default value 250  
    8. 20160114:14:30:03:005980 gpinitsystem:h93:wslu-[INFO]:-Checking configuration parameters, Completed  
    9. 20160114:14:30:04:005980 gpinitsystem:h93:wslu-[INFO]:-Commencing multi-home checks, please wait...  
    10. ..  
    11. 20160114:14:30:05:005980 gpinitsystem:h93:wslu-[INFO]:-Configuring build for standard array  
    12. 20160114:14:30:05:005980 gpinitsystem:h93:wslu-[INFO]:-Commencing multi-home checks, Completed  
    13. 20160114:14:30:05:005980 gpinitsystem:h93:wslu-[INFO]:-Building primary segment instance array, please wait...  
    14. ....  
    15. 20160114:14:30:08:005980 gpinitsystem:h93:wslu-[INFO]:-Building group mirror array type , please wait...  
    16. ....  
    17. 20160114:14:30:12:005980 gpinitsystem:h93:wslu-[INFO]:-Checking Master host  
    18. 20160114:14:30:12:005980 gpinitsystem:h93:wslu-[INFO]:-Checking new segment hosts, please wait...  
    19. ........  
    20. 20160114:14:30:28:005980 gpinitsystem:h93:wslu-[INFO]:-Checking new segment hosts, Completed  
    21. 20160114:14:30:28:005980 gpinitsystem:h93:wslu-[INFO]:-Greenplum Database Creation Parameters  
    22. 20160114:14:30:28:005980 gpinitsystem:h93:wslu-[INFO]:---------------------------------------  
    23. 20160114:14:30:28:005980 gpinitsystem:h93:wslu-[INFO]:-Master Configuration  
    24. 20160114:14:30:28:005980 gpinitsystem:h93:wslu-[INFO]:---------------------------------------  
    25. 20160114:14:30:28:005980 gpinitsystem:h93:wslu-[INFO]:-Master instance name       = EMC Greenplum DW  
    26. 20160114:14:30:28:005980 gpinitsystem:h93:wslu-[INFO]:-Master hostname            = h93  
    27. 20160114:14:30:28:005980 gpinitsystem:h93:wslu-[INFO]:-Master port                = 5432  
    28. 20160114:14:30:28:005980 gpinitsystem:h93:wslu-[INFO]:-Master instance dir        = /home/wslu/gp/gpsql/data/master/gpseg-1  
    29. 20160114:14:30:28:005980 gpinitsystem:h93:wslu-[INFO]:-Master LOCALE              = en_US.utf8  
    30. 20160114:14:30:29:005980 gpinitsystem:h93:wslu-[INFO]:-Greenplum segment prefix   = gpseg  
    31. 20160114:14:30:29:005980 gpinitsystem:h93:wslu-[INFO]:-Master Database            =  
    32. 20160114:14:30:29:005980 gpinitsystem:h93:wslu-[INFO]:-Master connections         = 250  
    33. 20160114:14:30:29:005980 gpinitsystem:h93:wslu-[INFO]:-Master buffers             = 128000kB  
    34. 20160114:14:30:29:005980 gpinitsystem:h93:wslu-[INFO]:-Segment connections        = 750  
    35. 20160114:14:30:29:005980 gpinitsystem:h93:wslu-[INFO]:-Segment buffers            = 128000kB  
    36. 20160114:14:30:29:005980 gpinitsystem:h93:wslu-[INFO]:-Checkpoint segments        = 8  
    37. 20160114:14:30:29:005980 gpinitsystem:h93:wslu-[INFO]:-Encoding                   = UNICODE  
    38. 20160114:14:30:29:005980 gpinitsystem:h93:wslu-[INFO]:-Postgres param file        = Off  
    39. 20160114:14:30:29:005980 gpinitsystem:h93:wslu-[INFO]:-Initdb to be used          = /home/wslu/gp/gpsql/bin/initdb  
    40. 20160114:14:30:29:005980 gpinitsystem:h93:wslu-[INFO]:-GP_LIBRARY_PATH is         = /home/wslu/gp/gpsql/lib  
    41. 20160114:14:30:29:005980 gpinitsystem:h93:wslu-[INFO]:-Ulimit check               = Passed  
    42. 20160114:14:30:29:005980 gpinitsystem:h93:wslu-[INFO]:-Array host connect type    = Single hostname per node  
    43. 20160114:14:30:29:005980 gpinitsystem:h93:wslu-[INFO]:-Master IP address [1]      = ::1  
    44. 20160114:14:30:29:005980 gpinitsystem:h93:wslu-[INFO]:-Master IP address [2]      = 192.168.4.93  
    45. 20160114:14:30:29:005980 gpinitsystem:h93:wslu-[INFO]:-Master IP address [3]      = fe80::225:90ff:fe3b:86c2  
    46. 20160114:14:30:29:005980 gpinitsystem:h93:wslu-[INFO]:-Standby Master             = Not Configured  
    47. 20160114:14:30:29:005980 gpinitsystem:h93:wslu-[INFO]:-Primary segment #          = 2  
    48. 20160114:14:30:29:005980 gpinitsystem:h93:wslu-[INFO]:-Total Database segments    = 4  
    49. 20160114:14:30:29:005980 gpinitsystem:h93:wslu-[INFO]:-Trusted shell              = ssh  
    50. 20160114:14:30:29:005980 gpinitsystem:h93:wslu-[INFO]:-Number segment hosts       = 2  
    51. 20160114:14:30:29:005980 gpinitsystem:h93:wslu-[INFO]:-Mirror port base           = 50000  
    52. 20160114:14:30:29:005980 gpinitsystem:h93:wslu-[INFO]:-Replicaton port base       = 41000  
    53. 20160114:14:30:29:005980 gpinitsystem:h93:wslu-[INFO]:-Mirror replicaton port base= 51000  
    54. 20160114:14:30:29:005980 gpinitsystem:h93:wslu-[INFO]:-Mirror segment #           = 2  
    55. 20160114:14:30:29:005980 gpinitsystem:h93:wslu-[INFO]:-Mirroring config           = ON  
    56. 20160114:14:30:29:005980 gpinitsystem:h93:wslu-[INFO]:-Mirroring type             = Group  
    57. 20160114:14:30:29:005980 gpinitsystem:h93:wslu-[INFO]:----------------------------------------  
    58. 20160114:14:30:29:005980 gpinitsystem:h93:wslu-[INFO]:-Greenplum Primary Segment Configuration  
    59. 20160114:14:30:29:005980 gpinitsystem:h93:wslu-[INFO]:----------------------------------------  
    60. 20160114:14:30:29:005980 gpinitsystem:h93:wslu-[INFO]:-h93      /home/wslu/gp/gpsql/data/primary/gpseg0        40000          2          0       41000  
    61. 20160114:14:30:29:005980 gpinitsystem:h93:wslu-[INFO]:-h93      /home/wslu/gp/gpsql/data/primary/gpseg1        40001          3          1       41001  
    62. 20160114:14:30:29:005980 gpinitsystem:h93:wslu-[INFO]:-h94      /home/wslu/gp/gpsql/data/primary/gpseg2        40000          4          2       41000  
    63. 20160114:14:30:29:005980 gpinitsystem:h93:wslu-[INFO]:-h94      /home/wslu/gp/gpsql/data/primary/gpseg3        40001          5          3       41001  
    64. 20160114:14:30:29:005980 gpinitsystem:h93:wslu-[INFO]:---------------------------------------  
    65. 20160114:14:30:29:005980 gpinitsystem:h93:wslu-[INFO]:-Greenplum Mirror Segment Configuration  
    66. 20160114:14:30:29:005980 gpinitsystem:h93:wslu-[INFO]:---------------------------------------  
    67. 20160114:14:30:29:005980 gpinitsystem:h93:wslu-[INFO]:-h94      /home/wslu/gp/gpsql/data/mirror/gpseg0          50000          6          0       51000  
    68. 20160114:14:30:29:005980 gpinitsystem:h93:wslu-[INFO]:-h94      /home/wslu/gp/gpsql/data/mirror/gpseg1          50001          7          1       51001  
    69. 20160114:14:30:29:005980 gpinitsystem:h93:wslu-[INFO]:-h93      /home/wslu/gp/gpsql/data/mirror/gpseg2          50000          8          2       51000  
    70. 20160114:14:30:29:005980 gpinitsystem:h93:wslu-[INFO]:-h93      /home/wslu/gp/gpsql/data/mirror/gpseg3          50001          9          3       51001  
    71. Continue with Greenplum creation Yy/Nn>  
    72. y  
    73. 20160114:14:30:32:005980 gpinitsystem:h93:wslu-[INFO]:-Building the Master instance database, please wait...  
    74. 20160114:14:31:08:005980 gpinitsystem:h93:wslu-[INFO]:-Starting the Master in admin mode  
    75. 20160114:14:32:01:005980 gpinitsystem:h93:wslu-[INFO]:-Commencing parallel build of primary segment instances  
    76. 20160114:14:32:01:005980 gpinitsystem:h93:wslu-[INFO]:-Spawning parallel processes    batch [1], please wait...  
    77. ....  
    78. 20160114:14:32:02:005980 gpinitsystem:h93:wslu-[INFO]:-Waiting for parallel processes batch [1], please wait...  
    79. ...........................................................  
    80. 20160114:14:33:01:005980 gpinitsystem:h93:wslu-[INFO]:------------------------------------------------  
    81. 20160114:14:33:01:005980 gpinitsystem:h93:wslu-[INFO]:-Parallel process exit status  
    82. 20160114:14:33:01:005980 gpinitsystem:h93:wslu-[INFO]:------------------------------------------------  
    83. 20160114:14:33:01:005980 gpinitsystem:h93:wslu-[INFO]:-Total processes marked as completed           = 4  
    84. 20160114:14:33:01:005980 gpinitsystem:h93:wslu-[INFO]:-Total processes marked as killed              = 0  
    85. 20160114:14:33:01:005980 gpinitsystem:h93:wslu-[INFO]:-Total processes marked as failed              = 0  
    86. 20160114:14:33:01:005980 gpinitsystem:h93:wslu-[INFO]:------------------------------------------------  
    87. 20160114:14:33:01:005980 gpinitsystem:h93:wslu-[INFO]:-Commencing parallel build of mirror segment instances  
    88. 20160114:14:33:01:005980 gpinitsystem:h93:wslu-[INFO]:-Spawning parallel processes    batch [1], please wait...  
    89. ....  
    90. 20160114:14:33:01:005980 gpinitsystem:h93:wslu-[INFO]:-Waiting for parallel processes batch [1], please wait...  
    91. .........................................  
    92. 20160114:14:33:43:005980 gpinitsystem:h93:wslu-[INFO]:------------------------------------------------  
    93. 20160114:14:33:43:005980 gpinitsystem:h93:wslu-[INFO]:-Parallel process exit status  
    94. 20160114:14:33:43:005980 gpinitsystem:h93:wslu-[INFO]:------------------------------------------------  
    95. 20160114:14:33:43:005980 gpinitsystem:h93:wslu-[INFO]:-Total processes marked as completed           = 4  
    96. 20160114:14:33:43:005980 gpinitsystem:h93:wslu-[INFO]:-Total processes marked as killed              = 0  
    97. 20160114:14:33:43:005980 gpinitsystem:h93:wslu-[INFO]:-Total processes marked as failed              = 0  
    98. 20160114:14:33:43:005980 gpinitsystem:h93:wslu-[INFO]:------------------------------------------------  
    99. 20160114:14:33:43:005980 gpinitsystem:h93:wslu-[INFO]:-Deleting distributed backout files  
    100. 20160114:14:33:43:005980 gpinitsystem:h93:wslu-[INFO]:-Removing back out file  
    101. 20160114:14:33:43:005980 gpinitsystem:h93:wslu-[INFO]:-No errors generated from parallel processes  
    102. 20160114:14:33:43:005980 gpinitsystem:h93:wslu-[INFO]:-Restarting the Greenplum instance in production mode  
    103. 20160114:14:33:43:001932 gpstop:h93:wslu-[INFO]:-Starting gpstop with args: -a -i -m -d /home/wslu/gp/gpsql/data/master/gpseg-1  
    104. 20160114:14:33:43:001932 gpstop:h93:wslu-[INFO]:-Gathering information and validating the environment...  
    105. 20160114:14:33:43:001932 gpstop:h93:wslu-[INFO]:-Obtaining Greenplum Master catalog information  
    106. 20160114:14:33:43:001932 gpstop:h93:wslu-[INFO]:-Obtaining Segment details from master...  
    107. 20160114:14:33:43:001932 gpstop:h93:wslu-[INFO]:-Greenplum Version: 'postgres (Greenplum Database) 4.3.99.00 build dev'  
    108. 20160114:14:33:43:001932 gpstop:h93:wslu-[INFO]:-There are 0 connections to the database  
    109. 20160114:14:33:43:001932 gpstop:h93:wslu-[INFO]:-Commencing Master instance shutdown with mode='immediate'  
    110. 20160114:14:33:43:001932 gpstop:h93:wslu-[INFO]:-Master host=h93  
    111. 20160114:14:33:43:001932 gpstop:h93:wslu-[INFO]:-Commencing Master instance shutdown with mode=immediate  
    112. 20160114:14:33:43:001932 gpstop:h93:wslu-[INFO]:-Master segment instance directory=/home/wslu/gp/gpsql/data/master/gpseg-1  
    113. 20160114:14:33:44:001932 gpstop:h93:wslu-[INFO]:-Attempting forceful termination of any leftover master process  
    114. 20160114:14:33:44:001932 gpstop:h93:wslu-[INFO]:-Terminating processes for segment /home/wslu/gp/gpsql/data/master/gpseg-1  
    115. 20160114:14:33:45:002019 gpstart:h93:wslu-[INFO]:-Starting gpstart with args: -a -d /home/wslu/gp/gpsql/data/master/gpseg-1  
    116. 20160114:14:33:45:002019 gpstart:h93:wslu-[INFO]:-Gathering information and validating the environment...  
    117. 20160114:14:33:45:002019 gpstart:h93:wslu-[INFO]:-Greenplum Binary Version: 'postgres (Greenplum Database) 4.3.99.00 build dev'  
    118. 20160114:14:33:45:002019 gpstart:h93:wslu-[INFO]:-Greenplum Catalog Version: '300701081'  
    119. 20160114:14:33:45:002019 gpstart:h93:wslu-[INFO]:-Starting Master instance in admin mode  
    120. 20160114:14:33:46:002019 gpstart:h93:wslu-[INFO]:-Obtaining Greenplum Master catalog information  
    121. 20160114:14:33:46:002019 gpstart:h93:wslu-[INFO]:-Obtaining Segment details from master...  
    122. 20160114:14:33:46:002019 gpstart:h93:wslu-[INFO]:-Setting new master era  
    123. 20160114:14:33:46:002019 gpstart:h93:wslu-[INFO]:-Master Started...  
    124. 20160114:14:33:46:002019 gpstart:h93:wslu-[INFO]:-Shutting down master  
    125. 20160114:14:33:47:002019 gpstart:h93:wslu-[INFO]:-Commencing parallel primary and mirror segment instance startup, please wait...  
    126. ........  
    127. 20160114:14:33:55:002019 gpstart:h93:wslu-[INFO]:-Process results...  
    128. 20160114:14:33:55:002019 gpstart:h93:wslu-[INFO]:-----------------------------------------------------  
    129. 20160114:14:33:55:002019 gpstart:h93:wslu-[INFO]:-   Successful segment starts                                            = 8  
    130. 20160114:14:33:55:002019 gpstart:h93:wslu-[INFO]:-   Failed segment starts                                                = 0  
    131. 20160114:14:33:55:002019 gpstart:h93:wslu-[INFO]:-   Skipped segment starts (segments are marked down in configuration)   = 0  
    132. 20160114:14:33:55:002019 gpstart:h93:wslu-[INFO]:-----------------------------------------------------  
    133. 20160114:14:33:55:002019 gpstart:h93:wslu-[INFO]:-  
    134. 20160114:14:33:55:002019 gpstart:h93:wslu-[INFO]:-Successfully started 8 of 8 segment instances  
    135. 20160114:14:33:55:002019 gpstart:h93:wslu-[INFO]:-----------------------------------------------------  
    136. 20160114:14:33:55:002019 gpstart:h93:wslu-[INFO]:-Starting Master instance h93 directory /home/wslu/gp/gpsql/data/master/gpseg-1  
    137. 20160114:14:33:56:002019 gpstart:h93:wslu-[INFO]:-Command pg_ctl reports Master h93 instance active  
    138. 20160114:14:33:56:002019 gpstart:h93:wslu-[INFO]:-No standby master configured.  skipping...  
    139. 20160114:14:33:56:002019 gpstart:h93:wslu-[INFO]:-Database successfully started  
    140. 20160114:14:33:59:005980 gpinitsystem:h93:wslu-[INFO]:-Completed restart of Greenplum instance in production mode  
    141. 20160114:14:33:59:005980 gpinitsystem:h93:wslu-[INFO]:-Loading gp_toolkit...  
    142. 20160114:14:34:02:005980 gpinitsystem:h93:wslu-[INFO]:-Scanning utility log file for any warning messages  
    143. 20160114:14:34:02:005980 gpinitsystem:h93:wslu-[INFO]:-Log file scan check passed  
    144. 20160114:14:34:02:005980 gpinitsystem:h93:wslu-[INFO]:-Greenplum Database instance successfully created  
    145. 20160114:14:34:02:005980 gpinitsystem:h93:wslu-[INFO]:-------------------------------------------------------  
    146. 20160114:14:34:02:005980 gpinitsystem:h93:wslu-[INFO]:-To complete the environment configuration, please  
    147. 20160114:14:34:02:005980 gpinitsystem:h93:wslu-[INFO]:-update wslu .bashrc file with the following  
    148. 20160114:14:34:02:005980 gpinitsystem:h93:wslu-[INFO]:-1. Ensure that the greenplum_path.sh file is sourced  
    149. 20160114:14:34:02:005980 gpinitsystem:h93:wslu-[INFO]:-2. Add "export MASTER_DATA_DIRECTORY=/home/wslu/gp/gpsql/data/master/gpseg-1"  
    150. 20160114:14:34:02:005980 gpinitsystem:h93:wslu-[INFO]:-   to access the Greenplum scripts for this instance:  
    151. 20160114:14:34:02:005980 gpinitsystem:h93:wslu-[INFO]:-   or, use -d /home/wslu/gp/gpsql/data/master/gpseg-1 option for the Greenplum scripts  
    152. 20160114:14:34:02:005980 gpinitsystem:h93:wslu-[INFO]:-   Example gpstate -d /home/wslu/gp/gpsql/data/master/gpseg-1  
    153. 20160114:14:34:02:005980 gpinitsystem:h93:wslu-[INFO]:-Script log file = /home/wslu/gpAdminLogs/gpinitsystem_20160114.log  
    154. 20160114:14:34:02:005980 gpinitsystem:h93:wslu-[INFO]:-To remove instance, run gpdeletesystem utility  
    155. 20160114:14:34:02:005980 gpinitsystem:h93:wslu-[INFO]:-To initialize a Standby Master Segment for this Greenplum instance  
    156. 20160114:14:34:02:005980 gpinitsystem:h93:wslu-[INFO]:-Review options for gpinitstandby  
    157. 20160114:14:34:02:005980 gpinitsystem:h93:wslu-[INFO]:-------------------------------------------------------  
    158. 20160114:14:34:02:005980 gpinitsystem:h93:wslu-[INFO]:-The Master /home/wslu/gp/gpsql/data/master/gpseg-1/pg_hba.conf post gpinitsystem  
    159. 20160114:14:34:02:005980 gpinitsystem:h93:wslu-[INFO]:-has been configured to allow all hosts within this new  
    160. 20160114:14:34:02:005980 gpinitsystem:h93:wslu-[INFO]:-array to intercommunicate. Any hosts external to this  
    161. 20160114:14:34:02:005980 gpinitsystem:h93:wslu-[INFO]:-new array must be explicitly added to this file  
    162. 20160114:14:34:02:005980 gpinitsystem:h93:wslu-[INFO]:-Refer to the Greenplum Admin support guide which is  
    163. 20160114:14:34:02:005980 gpinitsystem:h93:wslu-[INFO]:-located in the /home/wslu/gp/gpsql/docs directory  
    164. 20160114:14:34:02:005980 gpinitsystem:h93:wslu-[INFO]:-------------------------------------------------------  
    165. [wslu@h93 gpsql]$  

    （5）       查看目录结构：

    [python] view plain copy
     
    1. [wslu@h93 gpsql]$ ls data  
    2. master  mirror  primary  
    3. [wslu@h93 gpsql]$ ls data/master/  
    4. gpseg-1  
    5. [wslu@h93 gpsql]$ ls data/mirror/  
    6. gpseg2  gpseg3  
    7. [wslu@h93 gpsql]$ ls data/primary/  
    8. gpseg0  gpseg1  
    9. [wslu@h93 gpsql]$  
    10.    
    11. [wslu@h94 gpsql]$ ls data/  
    12. master  mirror  primary  
    13. [wslu@h94 gpsql]$ ls data/master/  
    14. [wslu@h94 gpsql]$ ls data/primary/  
    15. gpseg2  gpseg3  
    16. [wslu@h94 gpsql]$ ls data/mirror/  
    17. gpseg0  gpseg1  
    18. [wslu@h94 gpsql]$  

    （6）       在h94初始化备master（主备master必须在不同主机，如果要配置单机多节点，则不能配置备master。这是因为目前主备master必须在相同目录，所以必然不同主机，如果端口不是5432，那么需要指定PGPORT）：

    [python] view plain copy
     
    1. [wslu@h93 gpsql]$ PGPORT=5432 PGDATABASE=postgres gpinitstandby -s h94  
    2. 20160114:14:40:47:003933 gpinitstandby:h93:wslu-[INFO]:-Validating environment and parameters for standby initialization...  
    3. 20160114:14:40:47:003933 gpinitstandby:h93:wslu-[INFO]:-Checking for filespace directory /home/wslu/gp/gpsql/data/master/gpseg-1 on h94  
    4. 20160114:14:40:47:003933 gpinitstandby:h93:wslu-[INFO]:------------------------------------------------------  
    5. 20160114:14:40:47:003933 gpinitstandby:h93:wslu-[INFO]:-Greenplum standby master initialization parameters  
    6. 20160114:14:40:47:003933 gpinitstandby:h93:wslu-[INFO]:------------------------------------------------------  
    7. 20160114:14:40:47:003933 gpinitstandby:h93:wslu-[INFO]:-Greenplum master hostname               = h93  
    8. 20160114:14:40:47:003933 gpinitstandby:h93:wslu-[INFO]:-Greenplum master data directory         = /home/wslu/gp/gpsql/data/master/gpseg-1  
    9. 20160114:14:40:47:003933 gpinitstandby:h93:wslu-[INFO]:-Greenplum master port                   = 5432  
    10. 20160114:14:40:47:003933 gpinitstandby:h93:wslu-[INFO]:-Greenplum standby master hostname       = h94  
    11. 20160114:14:40:47:003933 gpinitstandby:h93:wslu-[INFO]:-Greenplum standby master port           = 5432  
    12. 20160114:14:40:47:003933 gpinitstandby:h93:wslu-[INFO]:-Greenplum standby master data directory = /home/wslu/gp/gpsql/data/master/gpseg-1  
    13. 20160114:14:40:47:003933 gpinitstandby:h93:wslu-[INFO]:-Greenplum update system catalog         = On  
    14. 20160114:14:40:47:003933 gpinitstandby:h93:wslu-[INFO]:------------------------------------------------------  
    15. 20160114:14:40:47:003933 gpinitstandby:h93:wslu-[INFO]:- Filespace locations  
    16. 20160114:14:40:47:003933 gpinitstandby:h93:wslu-[INFO]:------------------------------------------------------  
    17. 20160114:14:40:47:003933 gpinitstandby:h93:wslu-[INFO]:-pg_system -> /home/wslu/gp/gpsql/data/master/gpseg-1  
    18. Do you want to continue with standby master initialization? Yy|Nn (default=N):  
    19. > y  
    20. 20160114:14:40:53:003933 gpinitstandby:h93:wslu-[INFO]:-Syncing Greenplum Database extensions to standby  
    21. 20160114:14:40:53:003933 gpinitstandby:h93:wslu-[INFO]:-The packages on h94 are consistent.  
    22. 20160114:14:40:53:003933 gpinitstandby:h93:wslu-[INFO]:-Adding standby master to catalog...  
    23. 20160114:14:40:53:003933 gpinitstandby:h93:wslu-[INFO]:-Database catalog updated successfully.  
    24. 20160114:14:40:54:003933 gpinitstandby:h93:wslu-[INFO]:-Updating pg_hba.conf file...  
    25. 20160114:14:41:00:003933 gpinitstandby:h93:wslu-[INFO]:-pg_hba.conf files updated successfully.  
    26. 20160114:14:41:09:003933 gpinitstandby:h93:wslu-[INFO]:-Updating filespace flat files...  
    27. 20160114:14:41:09:003933 gpinitstandby:h93:wslu-[INFO]:-Filespace flat file updated successfully.  
    28. 20160114:14:41:10:003933 gpinitstandby:h93:wslu-[INFO]:-Starting standby master  
    29. 20160114:14:41:10:003933 gpinitstandby:h93:wslu-[INFO]:-Checking if standby master is running on host: h94  in directory: /home/wslu/gp/gpsql/data/master/gpseg-1  
    30. 20160114:14:41:11:003933 gpinitstandby:h93:wslu-[INFO]:-Cleaning up pg_hba.conf backup files...  
    31. 20160114:14:41:17:003933 gpinitstandby:h93:wslu-[INFO]:-Backup files of pg_hba.conf cleaned up successfully.  
    32. 20160114:14:41:17:003933 gpinitstandby:h93:wslu-[INFO]:-Successfully created standby master on h94  

    （7）       此时，h94的data/master目录就不为空了：

    [python] view plain copy
     
    1. $ [wslu@h94 gpsql]$ ls data/master/  
    2. gpseg-1  

    10. 测试

    [python] view plain copy
     
    1. [wslu@h93 gpsql]$ psql -p 5432 postgres  
    2. psql (8.3devel)  
    3. Type "help" for help.  
    4.    
    5. postgres=#  
    6. postgres=#  
    7. postgres=# db  
    8.         List of tablespaces  
    9.     Name    | Owner | Filespae Name  
    10. ------------+-------+---------------  
    11.  pg_default | wslu  | pg_system  
    12.  pg_global  | wslu  | pg_system  
    13. (2 rows)  
    至此，集群就初始化完成了。
    集群初始化完成后，默认会启动集群。
     
     
    ---------------------------------------------------------华丽的分割线-------------------------------------------------------------
    以下为Greenplum集群部分常用指令，以及关于配置单机多节点的一点说明。

    11. 启动集群

    以后，想手动启动集群，执行：

    [python] view plain copy
     
    1. $ gpstart –a  

    这里说一下，每次想使用集群的任何指令，必须执行：

    [python] view plain copy
     
    1. $ source greenplum-path.sh  
    2. $ exportMASTER_DATA_DIRECTORY=/home/wslu/gp/gpsql/data/master/gpseg-1  

    下文不再赘述。

    12. 停止集群

    [python] view plain copy
     
    1. $ gpstop –a  

    13. 重启集群

    [python] view plain copy
     
    1. $ gpstop –a –r  

    14. 查看集群状态

    [python] view plain copy
     
    1. $ gpstate –m | -e  

    15. reload配置文件

    在不停止集群情况下，若配置文件修改，reload配置文件：

    [python] view plain copy
     
    1. $ gpstop –u  

    16. 维护模式下启动master

    仅仅启动master来执行维护管理任务，不会影响segment中的数据。例如，在维护模式下你可以仅连接master实例的数据库并且编辑系统表设置。

    1.运行gpstart使用-m选项：

    [python] view plain copy
     
    1. $ gpstart –m  

    2.维护模式下连接master来维护系统表。例如：

    [python] view plain copy
     
    1. $ PGOPTIONS='-c gp_session_role=utility'psql template1  

    3.完成管理任务后，使master关闭工具模式。然后，重启进入正常模式：

    [python] view plain copy
     
    1. $ gpstop -m  

    17. 访问数据库

    可以使用psql连接集群：

    [python] view plain copy
     
    1.  [wslu@h93 gpsql]$ psql -p 5432 postgres  
    2. psql (8.3devel)  
    3. Type "help" for help.  
    4.    
    5. postgres=#  

    基本上常见命令与PostgreSQL的相同。可参考PostgreSQL官方手册和Greenplum官方手册。

    18. GUC参数配置

    使用gpconfig设置guc参数

    [python] view plain copy
     
    1. $ gpconfig -c gp_vmem_protect_limit -v4096MB  

    gpconfig可以设置master和所有segment的guc参数，也可以使用--masteronly参数只设置master的参数。设置完guc参数后需要根据guc参数类型决定重启集群或reload配置文件。

    显示guc参数

    [python] view plain copy
     
    1. $ psql –c ‘showstatement_mem;’ 或 gpconfig –show statement_mem  
    2. $ psql –c ‘show all;’ 或 gpconfig –l  

    19. 附录：关于配置单机多节点的说明

    如果要配置单机多节点，比如只想在h93配置2个主segment、2个镜像segment、1个master，只需要把hostfile_config中的h94删掉，然后在h93删除data/primary，data/mirror，data/master目录下的内容，重新初始化即可。
  • 相关阅读:
    序列化-请求数据校验。。。。。。
    python自动化测试-D5-学习笔记之二（常用模块之加密模块）
    python自动化测试-D5-学习笔记之二（常用模块之os,sys,random,string,time）
    python自动化测试-D5-学习笔记之二（常用模块之json模块）
    python自动化测试-D5-学习笔记之一（argv的使用）
    python自动化测试-D5-学习笔记之一（函数补充，内置函数，map，filter，eval）
    python习题：写一个函数打印两个字典中不一样的key和value
    python习题：用文件方式编写购物车程序，添加，查看和删除
    python习题：时间格式转换
    python习题：双色球
  • 原文地址：https://www.cnblogs.com/dhcn/p/7523192.html
Copyright © 2011-2022 走看看