软硬件环境：与上一篇文章一致；
集群中增加节点大致分为 4 个步骤：
1. 前期准备阶段；
2. 新节点加入集群（安装 GI 软件）；
3. 新节点安装 DB 软件；
4. 给新节点分配实例；
1.前期准备阶段
IP 配置、系统内核配置、安装 RPM 包、 SHELL 限制、分配用户和组、配置用户环境变量、目录创建及授权、配置互信等等一些前期环境准。
[root@vastdata2 ~]# mkdir -p /u01/app/grid mkdir -p /u01/app/11.2.0/grid mkdir -p /u01/app/oracle chown -R oracle:oinstall /u01 chown -R grid:oinstall /u01/app/grid chown -R grid:oinstall /u01/app/11.2.0 chmod -R 775 /u01
2.新节点加入集群（安装 GI 软件）
2.1对新节点做安装前的检查
[grid@vastdata4 ~]$ cluvfy stage -post nodedel -n vastdata3 -verbose
1 [grid@vastdata4 ~]$ cluvfy stage -post nodedel -n vastdata3 -verbose 2 3 Performing post-checks for node removal 4 5 Checking CRS integrity... 6 7 Clusterware version consistency passed 8 The Oracle Clusterware is healthy on node "vastdata4" 9 10 CRS integrity check passed 11 Result: 12 Node removal check passed 13 14 Post-check for node removal was successful. 15 [grid@vastdata4 ~]$ 16 [grid@vastdata4 ~]$ 17 [grid@vastdata4 ~]$ 18 [grid@vastdata4 ~]$ 19 [grid@vastdata4 ~]$ 20 [grid@vastdata4 ~]$ 21 [grid@vastdata4 ~]$ 22 [grid@vastdata4 ~]$ cluvfy stage -pre nodeadd -n vastdata3 -verbose 23 24 Performing pre-checks for node addition 25 26 Checking node reachability... 27 28 Check: Node reachability from node "vastdata4" 29 Destination Node Reachable? 30 ------------------------------------ ------------------------ 31 vastdata3 yes 32 Result: Node reachability check passed from node "vastdata4" 33 34 Checking user equivalence... 35 36 Check: User equivalence for user "grid" 37 Node Name Status 38 ------------------------------------ ------------------------ 39 vastdata3 passed 40 Result: User equivalence check passed for user "grid" 41 42 Checking CRS integrity... 43 44 Clusterware version consistency passed 45 The Oracle Clusterware is healthy on node "vastdata4" 46 47 CRS integrity check passed 48 49 Checking shared resources... 50 51 Checking CRS home location... 52 "/u01/app/11.2.0/grid" is shared 53 Result: Shared resources check for node addition passed 54 55 Checking node connectivity... 56 57 Checking hosts config file... 58 Node Name Status 59 ------------------------------------ ------------------------ 60 vastdata3 passed 61 vastdata4 passed 62 63 Verification of the hosts config file successful 64 65 Interface information for node "vastdata3" 66 Name IP Address Subnet Gateway Def. Gateway HW Address MTU 67 ------ --------------- --------------- --------------- --------------- ----------------- ------ 68 eth0 192.168.0.11 192.168.0.0 0.0.0.0 UNKNOWN 08:00:27:D3:55:CB 1500 69 eth1 10.0.0.2 10.0.0.0 0.0.0.0 UNKNOWN 08:00:27:B4:C3:C7 1500 70 eth2 10.0.0.3 10.0.0.0 0.0.0.0 UNKNOWN 08:00:27:02:BD:9A 1500 71 72 Interface information for node "vastdata4" 73 Name IP Address Subnet Gateway Def. Gateway HW Address MTU 74 ------ --------------- --------------- --------------- --------------- ----------------- ------ 75 eth0 192.168.0.12 192.168.0.0 0.0.0.0 UNKNOWN 08:00:27:EF:F6:15 1500 76 eth0 192.168.0.33 192.168.0.0 0.0.0.0 UNKNOWN 08:00:27:EF:F6:15 1500 77 eth0 192.168.0.35 192.168.0.0 0.0.0.0 UNKNOWN 08:00:27:EF:F6:15 1500 78 eth0 192.168.0.23 192.168.0.0 0.0.0.0 UNKNOWN 08:00:27:EF:F6:15 1500 79 eth0 192.168.0.34 192.168.0.0 0.0.0.0 UNKNOWN 08:00:27:EF:F6:15 1500 80 eth1 10.0.0.4 10.0.0.0 0.0.0.0 UNKNOWN 08:00:27:82:3A:57 1500 81 eth1 169.254.17.92 169.254.0.0 0.0.0.0 UNKNOWN 08:00:27:82:3A:57 1500 82 eth2 10.0.0.5 10.0.0.0 0.0.0.0 UNKNOWN 08:00:27:4C:5D:39 1500 83 eth2 169.254.236.167 169.254.128.0 0.0.0.0 UNKNOWN 08:00:27:4C:5D:39 1500 84 85 Check: Node connectivity for interface "eth0" 86 Source Destination Connected? 87 ------------------------------ ------------------------------ ---------------- 88 vastdata3[192.168.0.11] vastdata4[192.168.0.12] yes 89 vastdata3[192.168.0.11] vastdata4[192.168.0.33] yes 90 vastdata3[192.168.0.11] vastdata4[192.168.0.35] yes 91 vastdata3[192.168.0.11] vastdata4[192.168.0.23] yes 92 vastdata3[192.168.0.11] vastdata4[192.168.0.34] yes 93 vastdata4[192.168.0.12] vastdata4[192.168.0.33] yes 94 vastdata4[192.168.0.12] vastdata4[192.168.0.35] yes 95 vastdata4[192.168.0.12] vastdata4[192.168.0.23] yes 96 vastdata4[192.168.0.12] vastdata4[192.168.0.34] yes 97 vastdata4[192.168.0.33] vastdata4[192.168.0.35] yes 98 vastdata4[192.168.0.33] vastdata4[192.168.0.23] yes 99 vastdata4[192.168.0.33] vastdata4[192.168.0.34] yes 100 vastdata4[192.168.0.35] vastdata4[192.168.0.23] yes 101 vastdata4[192.168.0.35] vastdata4[192.168.0.34] yes 102 vastdata4[192.168.0.23] vastdata4[192.168.0.34] yes 103 Result: Node connectivity passed for interface "eth0" 104 105 Check: TCP connectivity of subnet "192.168.0.0" 106 Source Destination Connected? 107 ------------------------------ ------------------------------ ---------------- 108 vastdata4:192.168.0.12 vastdata3:192.168.0.11 passed 109 vastdata4:192.168.0.12 vastdata4:192.168.0.33 passed 110 vastdata4:192.168.0.12 vastdata4:192.168.0.35 passed 111 vastdata4:192.168.0.12 vastdata4:192.168.0.23 passed 112 vastdata4:192.168.0.12 vastdata4:192.168.0.34 passed 113 Result: TCP connectivity check passed for subnet "192.168.0.0" 114 115 Check: Node connectivity for interface "eth1" 116 Source Destination Connected? 117 ------------------------------ ------------------------------ ---------------- 118 vastdata3[10.0.0.2] vastdata3[10.0.0.3] yes 119 vastdata3[10.0.0.2] vastdata4[10.0.0.4] yes 120 vastdata3[10.0.0.2] vastdata4[10.0.0.5] yes 121 vastdata3[10.0.0.3] vastdata4[10.0.0.4] yes 122 vastdata3[10.0.0.3] vastdata4[10.0.0.5] yes 123 vastdata4[10.0.0.4] vastdata4[10.0.0.5] yes 124 Result: Node connectivity passed for interface "eth1" 125 126 Check: TCP connectivity of subnet "10.0.0.0" 127 Source Destination Connected? 128 ------------------------------ ------------------------------ ---------------- 129 vastdata4:10.0.0.4 vastdata3:10.0.0.2 passed 130 vastdata4:10.0.0.4 vastdata3:10.0.0.3 passed 131 vastdata4:10.0.0.4 vastdata4:10.0.0.5 passed 132 Result: TCP connectivity check passed for subnet "10.0.0.0" 133 134 Check: Node connectivity for interface "eth2" 135 Checking subnet mask consistency... 136 Subnet mask consistency check passed for subnet "192.168.0.0". 137 Subnet mask consistency check passed for subnet "10.0.0.0". 138 Subnet mask consistency check passed. 139 140 Result: Node connectivity check passed 141 142 Checking multicast communication... 143 144 Checking subnet "192.168.0.0" for multicast communication with multicast group "230.0.1.0"... 145 Check of subnet "192.168.0.0" for multicast communication with multicast group "230.0.1.0" passed. 146 147 Checking subnet "10.0.0.0" for multicast communication with multicast group "230.0.1.0"... 148 Check of subnet "10.0.0.0" for multicast communication with multicast group "230.0.1.0" passed. 149 150 Check of multicast communication passed. 151 152 Check: Total memory 153 Node Name Available Required Status 154 ------------ ------------------------ ------------------------ ---------- 155 vastdata4 2.8198GB (2956724.0KB) 1.5GB (1572864.0KB) passed 156 vastdata3 2.8198GB (2956724.0KB) 1.5GB (1572864.0KB) passed 157 Result: Total memory check passed 158 159 Check: Available memory 160 Node Name Available Required Status 161 ------------ ------------------------ ------------------------ ---------- 162 vastdata4 1.3675GB (1433912.0KB) 50MB (51200.0KB) passed 163 vastdata3 2.6447GB (2773172.0KB) 50MB (51200.0KB) passed 164 Result: Available memory check passed 165 166 Check: Swap space 167 Node Name Available Required Status 168 ------------ ------------------------ ------------------------ ---------- 169 vastdata4 6GB (6291448.0KB) 2.8198GB (2956724.0KB) passed 170 vastdata3 6GB (6291448.0KB) 2.8198GB (2956724.0KB) passed 171 Result: Swap space check passed 172 173 Check: Free disk space for "vastdata4:/u01/app/11.2.0/grid,vastdata4:/tmp" 174 Path Node Name Mount point Available Required Status 175 ---------------- ------------ ------------ ------------ ------------ ------------ 176 /u01/app/11.2.0/grid vastdata4 / 5.2812GB 7.5GB failed 177 /tmp vastdata4 / 5.2812GB 7.5GB failed 178 Result: Free disk space check failed for "vastdata4:/u01/app/11.2.0/grid,vastdata4:/tmp" 179 180 Check: Free disk space for "vastdata3:/u01/app/11.2.0/grid,vastdata3:/tmp" 181 Path Node Name Mount point Available Required Status 182 ---------------- ------------ ------------ ------------ ------------ ------------ 183 /u01/app/11.2.0/grid vastdata3 / 15.7051GB 7.5GB passed 184 /tmp vastdata3 / 15.7051GB 7.5GB passed 185 Result: Free disk space check passed for "vastdata3:/u01/app/11.2.0/grid,vastdata3:/tmp" 186 187 Check: User existence for "grid" 188 Node Name Status Comment 189 ------------ ------------------------ ------------------------ 190 vastdata4 passed exists(1100) 191 vastdata3 passed exists(1100) 192 193 Checking for multiple users with UID value 1100 194 Result: Check for multiple users with UID value 1100 passed 195 Result: User existence check passed for "grid" 196 197 Check: Run level 198 Node Name run level Required Status 199 ------------ ------------------------ ------------------------ ---------- 200 vastdata4 5 3,5 passed 201 vastdata3 5 3,5 passed 202 Result: Run level check passed 203 204 Check: Hard limits for "maximum open file descriptors" 205 Node Name Type Available Required Status 206 ---------------- ------------ ------------ ------------ ---------------- 207 vastdata4 hard 65536 65536 passed 208 vastdata3 hard 65536 65536 passed 209 Result: Hard limits check passed for "maximum open file descriptors" 210 211 Check: Soft limits for "maximum open file descriptors" 212 Node Name Type Available Required Status 213 ---------------- ------------ ------------ ------------ ---------------- 214 vastdata4 soft 1024 1024 passed 215 vastdata3 soft 1024 1024 passed 216 Result: Soft limits check passed for "maximum open file descriptors" 217 218 Check: Hard limits for "maximum user processes" 219 Node Name Type Available Required Status 220 ---------------- ------------ ------------ ------------ ---------------- 221 vastdata4 hard 16384 16384 passed 222 vastdata3 hard 16384 16384 passed 223 Result: Hard limits check passed for "maximum user processes" 224 225 Check: Soft limits for "maximum user processes" 226 Node Name Type Available Required Status 227 ---------------- ------------ ------------ ------------ ---------------- 228 vastdata4 soft 2047 2047 passed 229 vastdata3 soft 2047 2047 passed 230 Result: Soft limits check passed for "maximum user processes" 231 232 Check: System architecture 233 Node Name Available Required Status 234 ------------ ------------------------ ------------------------ ---------- 235 vastdata4 x86_64 x86_64 passed 236 vastdata3 x86_64 x86_64 passed 237 Result: System architecture check passed 238 239 Check: Kernel version 240 Node Name Available Required Status 241 ------------ ------------------------ ------------------------ ---------- 242 vastdata4 2.6.32-431.el6.x86_64 2.6.32 passed 243 vastdata3 2.6.32-431.el6.x86_64 2.6.32 passed 244 Result: Kernel version check passed 245 246 Check: Kernel parameter for "semmsl" 247 Node Name Current Configured Required Status Comment 248 ---------------- ------------ ------------ ------------ ------------ ------------ 249 vastdata4 250 250 250 passed 250 vastdata3 250 250 250 passed 251 Result: Kernel parameter check passed for "semmsl" 252 253 Check: Kernel parameter for "semmns" 254 Node Name Current Configured Required Status Comment 255 ---------------- ------------ ------------ ------------ ------------ ------------ 256 vastdata4 32000 32000 32000 passed 257 vastdata3 32000 32000 32000 passed 258 Result: Kernel parameter check passed for "semmns" 259 260 Check: Kernel parameter for "semopm" 261 Node Name Current Configured Required Status Comment 262 ---------------- ------------ ------------ ------------ ------------ ------------ 263 vastdata4 100 100 100 passed 264 vastdata3 100 100 100 passed 265 Result: Kernel parameter check passed for "semopm" 266 267 Check: Kernel parameter for "semmni" 268 Node Name Current Configured Required Status Comment 269 ---------------- ------------ ------------ ------------ ------------ ------------ 270 vastdata4 128 128 128 passed 271 vastdata3 128 128 128 passed 272 Result: Kernel parameter check passed for "semmni" 273 274 Check: Kernel parameter for "shmmax" 275 Node Name Current Configured Required Status Comment 276 ---------------- ------------ ------------ ------------ ------------ ------------ 277 vastdata4 68719476736 68719476736 1513842688 passed 278 vastdata3 68719476736 68719476736 1513842688 passed 279 Result: Kernel parameter check passed for "shmmax" 280 281 Check: Kernel parameter for "shmmni" 282 Node Name Current Configured Required Status Comment 283 ---------------- ------------ ------------ ------------ ------------ ------------ 284 vastdata4 4096 4096 4096 passed 285 vastdata3 4096 4096 4096 passed 286 Result: Kernel parameter check passed for "shmmni" 287 288 Check: Kernel parameter for "shmall" 289 Node Name Current Configured Required Status Comment 290 ---------------- ------------ ------------ ------------ ------------ ------------ 291 vastdata4 4294967296 4294967296 2097152 passed 292 vastdata3 4294967296 4294967296 2097152 passed 293 Result: Kernel parameter check passed for "shmall" 294 295 Check: Kernel parameter for "file-max" 296 Node Name Current Configured Required Status Comment 297 ---------------- ------------ ------------ ------------ ------------ ------------ 298 vastdata4 6815744 6815744 6815744 passed 299 vastdata3 6815744 6815744 6815744 passed 300 Result: Kernel parameter check passed for "file-max" 301 302 Check: Kernel parameter for "ip_local_port_range" 303 Node Name Current Configured Required Status Comment 304 ---------------- ------------ ------------ ------------ ------------ ------------ 305 vastdata4 between 9000.0 & 65500.0 between 9000.0 & 65500.0 between 9000.0 & 65500.0 passed 306 vastdata3 between 9000.0 & 65500.0 between 9000.0 & 65500.0 between 9000.0 & 65500.0 passed 307 Result: Kernel parameter check passed for "ip_local_port_range" 308 309 Check: Kernel parameter for "rmem_default" 310 Node Name Current Configured Required Status Comment 311 ---------------- ------------ ------------ ------------ ------------ ------------ 312 vastdata4 262144 262144 262144 passed 313 vastdata3 262144 262144 262144 passed 314 Result: Kernel parameter check passed for "rmem_default" 315 316 Check: Kernel parameter for "rmem_max" 317 Node Name Current Configured Required Status Comment 318 ---------------- ------------ ------------ ------------ ------------ ------------ 319 vastdata4 4194304 4194304 4194304 passed 320 vastdata3 4194304 4194304 4194304 passed 321 Result: Kernel parameter check passed for "rmem_max" 322 323 Check: Kernel parameter for "wmem_default" 324 Node Name Current Configured Required Status Comment 325 ---------------- ------------ ------------ ------------ ------------ ------------ 326 vastdata4 262144 262144 262144 passed 327 vastdata3 262144 262144 262144 passed 328 Result: Kernel parameter check passed for "wmem_default" 329 330 Check: Kernel parameter for "wmem_max" 331 Node Name Current Configured Required Status Comment 332 ---------------- ------------ ------------ ------------ ------------ ------------ 333 vastdata4 1048576 1048576 1048576 passed 334 vastdata3 1048576 1048576 1048576 passed 335 Result: Kernel parameter check passed for "wmem_max" 336 337 Check: Kernel parameter for "aio-max-nr" 338 Node Name Current Configured Required Status Comment 339 ---------------- ------------ ------------ ------------ ------------ ------------ 340 vastdata4 1048576 1048576 1048576 passed 341 vastdata3 1048576 1048576 1048576 passed 342 Result: Kernel parameter check passed for "aio-max-nr" 343 344 Check: Package existence for "binutils" 345 Node Name Available Required Status 346 ------------ ------------------------ ------------------------ ---------- 347 vastdata4 binutils-2.20.51.0.2-5.36.el6 binutils-2.20.51.0.2 passed 348 vastdata3 binutils-2.20.51.0.2-5.36.el6 binutils-2.20.51.0.2 passed 349 Result: Package existence check passed for "binutils" 350 351 Check: Package existence for "compat-libcap1" 352 Node Name Available Required Status 353 ------------ ------------------------ ------------------------ ---------- 354 vastdata4 compat-libcap1-1.10-1 compat-libcap1-1.10 passed 355 vastdata3 compat-libcap1-1.10-1 compat-libcap1-1.10 passed 356 Result: Package existence check passed for "compat-libcap1" 357 358 Check: Package existence for "compat-libstdc++-33(x86_64)" 359 Node Name Available Required Status 360 ------------ ------------------------ ------------------------ ---------- 361 vastdata4 compat-libstdc++-33(x86_64)-3.2.3-69.el6 compat-libstdc++-33(x86_64)-3.2.3 passed 362 vastdata3 compat-libstdc++-33(x86_64)-3.2.3-69.el6 compat-libstdc++-33(x86_64)-3.2.3 passed 363 Result: Package existence check passed for "compat-libstdc++-33(x86_64)" 364 365 Check: Package existence for "libgcc(x86_64)" 366 Node Name Available Required Status 367 ------------ ------------------------ ------------------------ ---------- 368 vastdata4 libgcc(x86_64)-4.4.7-4.el6 libgcc(x86_64)-4.4.4 passed 369 vastdata3 libgcc(x86_64)-4.4.7-4.el6 libgcc(x86_64)-4.4.4 passed 370 Result: Package existence check passed for "libgcc(x86_64)" 371 372 Check: Package existence for "libstdc++(x86_64)" 373 Node Name Available Required Status 374 ------------ ------------------------ ------------------------ ---------- 375 vastdata4 libstdc++(x86_64)-4.4.7-4.el6 libstdc++(x86_64)-4.4.4 passed 376 vastdata3 libstdc++(x86_64)-4.4.7-4.el6 libstdc++(x86_64)-4.4.4 passed 377 Result: Package existence check passed for "libstdc++(x86_64)" 378 379 Check: Package existence for "libstdc++-devel(x86_64)" 380 Node Name Available Required Status 381 ------------ ------------------------ ------------------------ ---------- 382 vastdata4 libstdc++-devel(x86_64)-4.4.7-4.el6 libstdc++-devel(x86_64)-4.4.4 passed 383 vastdata3 libstdc++-devel(x86_64)-4.4.7-4.el6 libstdc++-devel(x86_64)-4.4.4 passed 384 Result: Package existence check passed for "libstdc++-devel(x86_64)" 385 386 Check: Package existence for "sysstat" 387 Node Name Available Required Status 388 ------------ ------------------------ ------------------------ ---------- 389 vastdata4 sysstat-9.0.4-22.el6 sysstat-9.0.4 passed 390 vastdata3 sysstat-9.0.4-22.el6 sysstat-9.0.4 passed 391 Result: Package existence check passed for "sysstat" 392 393 Check: Package existence for "gcc" 394 Node Name Available Required Status 395 ------------ ------------------------ ------------------------ ---------- 396 vastdata4 gcc-4.4.7-4.el6 gcc-4.4.4 passed 397 vastdata3 gcc-4.4.7-4.el6 gcc-4.4.4 passed 398 Result: Package existence check passed for "gcc" 399 400 Check: Package existence for "gcc-c++" 401 Node Name Available Required Status 402 ------------ ------------------------ ------------------------ ---------- 403 vastdata4 gcc-c++-4.4.7-4.el6 gcc-c++-4.4.4 passed 404 vastdata3 gcc-c++-4.4.7-4.el6 gcc-c++-4.4.4 passed 405 Result: Package existence check passed for "gcc-c++" 406 407 Check: Package existence for "ksh" 408 Node Name Available Required Status 409 ------------ ------------------------ ------------------------ ---------- 410 vastdata4 ksh-20120801-10.el6 ksh-20100621 passed 411 vastdata3 ksh-20120801-10.el6 ksh-20100621 passed 412 Result: Package existence check passed for "ksh" 413 414 Check: Package existence for "make" 415 Node Name Available Required Status 416 ------------ ------------------------ ------------------------ ---------- 417 vastdata4 make-3.81-20.el6 make-3.81 passed 418 vastdata3 make-3.81-20.el6 make-3.81 passed 419 Result: Package existence check passed for "make" 420 421 Check: Package existence for "glibc(x86_64)" 422 Node Name Available Required Status 423 ------------ ------------------------ ------------------------ ---------- 424 vastdata4 glibc(x86_64)-2.12-1.132.el6 glibc(x86_64)-2.12 passed 425 vastdata3 glibc(x86_64)-2.12-1.132.el6 glibc(x86_64)-2.12 passed 426 Result: Package existence check passed for "glibc(x86_64)" 427 428 Check: Package existence for "glibc-devel(x86_64)" 429 Node Name Available Required Status 430 ------------ ------------------------ ------------------------ ---------- 431 vastdata4 glibc-devel(x86_64)-2.12-1.132.el6 glibc-devel(x86_64)-2.12 passed 432 vastdata3 glibc-devel(x86_64)-2.12-1.132.el6 glibc-devel(x86_64)-2.12 passed 433 Result: Package existence check passed for "glibc-devel(x86_64)" 434 435 Check: Package existence for "libaio(x86_64)" 436 Node Name Available Required Status 437 ------------ ------------------------ ------------------------ ---------- 438 vastdata4 libaio(x86_64)-0.3.107-10.el6 libaio(x86_64)-0.3.107 passed 439 vastdata3 libaio(x86_64)-0.3.107-10.el6 libaio(x86_64)-0.3.107 passed 440 Result: Package existence check passed for "libaio(x86_64)" 441 442 Check: Package existence for "libaio-devel(x86_64)" 443 Node Name Available Required Status 444 ------------ ------------------------ ------------------------ ---------- 445 vastdata4 libaio-devel(x86_64)-0.3.107-10.el6 libaio-devel(x86_64)-0.3.107 passed 446 vastdata3 libaio-devel(x86_64)-0.3.107-10.el6 libaio-devel(x86_64)-0.3.107 passed 447 Result: Package existence check passed for "libaio-devel(x86_64)" 448 449 Checking for multiple users with UID value 0 450 Result: Check for multiple users with UID value 0 passed 451 452 Check: Current group ID 453 Result: Current group ID check passed 454 455 Starting check for consistency of primary group of root user 456 Node Name Status 457 ------------------------------------ ------------------------ 458 vastdata4 passed 459 vastdata3 passed 460 461 Check for consistency of root user's primary group passed 462 463 Checking OCR integrity... 464 465 OCR integrity check passed 466 467 Checking Oracle Cluster Voting Disk configuration... 468 469 Oracle Cluster Voting Disk configuration check passed 470 Check: Time zone consistency 471 Result: Time zone consistency check passed 472 473 Starting Clock synchronization checks using Network Time Protocol(NTP)... 474 475 NTP Configuration file check started... 476 Network Time Protocol(NTP) configuration file not found on any of the nodes. Oracle Cluster Time Synchronization Service(CTSS) can be used instead of NTP for time synchronization on the cluster nodes 477 No NTP Daemons or Services were found to be running 478 479 Result: Clock synchronization check using Network Time Protocol(NTP) passed 480 481 Checking to make sure user "grid" is not in "root" group 482 Node Name Status Comment 483 ------------ ------------------------ ------------------------ 484 vastdata4 passed does not exist 485 vastdata3 passed does not exist 486 Result: User "grid" is not part of "root" group. Check passed 487 Checking consistency of file "/etc/resolv.conf" across nodes 488 489 Checking the file "/etc/resolv.conf" to make sure only one of domain and search entries is defined 490 File "/etc/resolv.conf" does not have both domain and search entries defined 491 Checking if domain entry in file "/etc/resolv.conf" is consistent across the nodes... 492 domain entry in file "/etc/resolv.conf" is consistent across nodes 493 Checking if search entry in file "/etc/resolv.conf" is consistent across the nodes... 494 search entry in file "/etc/resolv.conf" is consistent across nodes 495 Checking file "/etc/resolv.conf" to make sure that only one search entry is defined 496 All nodes have one search entry defined in file "/etc/resolv.conf" 497 Checking all nodes to make sure that search entry is "us.oracle.com" as found on node "vastdata4" 498 All nodes of the cluster have same value for 'search' 499 Checking DNS response time for an unreachable node 500 Node Name Status 501 ------------------------------------ ------------------------ 502 vastdata4 passed 503 vastdata3 passed 504 The DNS response time for an unreachable node is within acceptable limit on all nodes 505 506 File "/etc/resolv.conf" is consistent across nodes 507 508 Checking integrity of name service switch configuration file "/etc/nsswitch.conf" ... 509 Checking if "hosts" entry in file "/etc/nsswitch.conf" is consistent across nodes... 510 Checking file "/etc/nsswitch.conf" to make sure that only one "hosts" entry is defined 511 More than one "hosts" entry does not exist in any "/etc/nsswitch.conf" file 512 Check for integrity of name service switch configuration file "/etc/nsswitch.conf" passed 513 514 Pre-check for node addition was successful.
2.2检查集群内节点状态
[grid@vastdata4 ~]$ olsnodes -t vastdata4 Unpinned
2.3给新的节点安装 GI 软件
[grid@vastdata4 ~]$ /u01/app/11.2.0/grid/oui/bin/addNode.sh -silent "CLUSTER_NEW_NODES={vastdata3}" "CLUSTER_NEW_VIRTUAL_HOSTNAMES={vastdata3-vip}"
1 [grid@vastdata4 ~]$ /u01/app/11.2.0/grid/oui/bin/addNode.sh -silent "CLUSTER_NEW_NODES={vastdata3}" "CLUSTER_NEW_VIRTUAL_HOSTNAMES={vastdata3-vip}" 2 3 Performing pre-checks for node addition 4 5 Checking node reachability... 6 Node reachability check passed from node "vastdata4" 7 8 Checking user equivalence... 9 User equivalence check passed for user "grid" 10 11 Checking CRS integrity... 12 13 Clusterware version consistency passed 14 15 CRS integrity check passed 16 17 Checking shared resources... 18 19 Checking CRS home location... 20 "/u01/app/11.2.0/grid" is shared 21 Shared resources check for node addition passed 22 23 Checking node connectivity... 24 25 Checking hosts config file... 26 27 Verification of the hosts config file successful 28 29 Check: Node connectivity for interface "eth0" 30 Node connectivity passed for interface "eth0" 31 TCP connectivity check passed for subnet "192.168.0.0" 32 Check: Node connectivity for interface "eth1" 33 Node connectivity passed for interface "eth1" 34 TCP connectivity check passed for subnet "10.0.0.0" 35 36 Check: Node connectivity for interface "eth2" 37 Checking subnet mask consistency... 38 Subnet mask consistency check passed for subnet "192.168.0.0". 39 Subnet mask consistency check passed for subnet "10.0.0.0". 40 Subnet mask consistency check passed. 41 42 Node connectivity check passed 43 44 Checking multicast communication... 45 46 Checking subnet "192.168.0.0" for multicast communication with multicast group "230.0.1.0"... 47 Check of subnet "192.168.0.0" for multicast communication with multicast group "230.0.1.0" passed. 48 49 Checking subnet "10.0.0.0" for multicast communication with multicast group "230.0.1.0"... 50 Check of subnet "10.0.0.0" for multicast communication with multicast group "230.0.1.0" passed. 51 52 Check of multicast communication passed. 53 Total memory check passed 54 Available memory check passed 55 Swap space check passed 56 Free disk space check failed for "vastdata4:/u01/app/11.2.0/grid,vastdata4:/tmp" 57 Check failed on nodes: 58 vastdata4 59 Free disk space check passed for "vastdata3:/u01/app/11.2.0/grid,vastdata3:/tmp" 60 Check for multiple users with UID value 1100 passed 61 User existence check passed for "grid" 62 Run level check passed 63 Hard limits check passed for "maximum open file descriptors" 64 Soft limits check passed for "maximum open file descriptors" 65 Hard limits check passed for "maximum user processes" 66 Soft limits check passed for "maximum user processes" 67 System architecture check passed 68 Kernel version check passed 69 Kernel parameter check passed for "semmsl" 70 Kernel parameter check passed for "semmns" 71 Kernel parameter check passed for "semopm" 72 Kernel parameter check passed for "semmni" 73 Kernel parameter check passed for "shmmax" 74 Kernel parameter check passed for "shmmni" 75 Kernel parameter check passed for "shmall" 76 Kernel parameter check passed for "file-max" 77 Kernel parameter check passed for "ip_local_port_range" 78 Kernel parameter check passed for "rmem_default" 79 Kernel parameter check passed for "rmem_max" 80 Kernel parameter check passed for "wmem_default" 81 Kernel parameter check passed for "wmem_max" 82 Kernel parameter check passed for "aio-max-nr" 83 Package existence check passed for "binutils" 84 Package existence check passed for "compat-libcap1" 85 Package existence check passed for "compat-libstdc++-33(x86_64)" 86 Package existence check passed for "libgcc(x86_64)" 87 Package existence check passed for "libstdc++(x86_64)" 88 Package existence check passed for "libstdc++-devel(x86_64)" 89 Package existence check passed for "sysstat" 90 Package existence check passed for "gcc" 91 Package existence check passed for "gcc-c++" 92 Package existence check passed for "ksh" 93 Package existence check passed for "make" 94 Package existence check passed for "glibc(x86_64)" 95 Package existence check passed for "glibc-devel(x86_64)" 96 Package existence check passed for "libaio(x86_64)" 97 Package existence check passed for "libaio-devel(x86_64)" 98 Check for multiple users with UID value 0 passed 99 Current group ID check passed 100 101 Starting check for consistency of primary group of root user 102 103 Check for consistency of root user's primary group passed 104 105 Checking OCR integrity... 106 107 OCR integrity check passed 108 109 Checking Oracle Cluster Voting Disk configuration... 110 111 Oracle Cluster Voting Disk configuration check passed 112 Time zone consistency check passed 113 114 Starting Clock synchronization checks using Network Time Protocol(NTP)... 115 116 NTP Configuration file check started... 117 No NTP Daemons or Services were found to be running 118 119 Clock synchronization check using Network Time Protocol(NTP) passed 120 121 User "grid" is not part of "root" group. Check passed 122 Checking consistency of file "/etc/resolv.conf" across nodes 123 124 File "/etc/resolv.conf" does not have both domain and search entries defined 125 domain entry in file "/etc/resolv.conf" is consistent across nodes 126 search entry in file "/etc/resolv.conf" is consistent across nodes 127 All nodes have one search entry defined in file "/etc/resolv.conf" 128 The DNS response time for an unreachable node is within acceptable limit on all nodes 129 130 File "/etc/resolv.conf" is consistent across nodes 131 132 Checking integrity of name service switch configuration file "/etc/nsswitch.conf" ... 133 Check for integrity of name service switch configuration file "/etc/nsswitch.conf" passed 134 135 Checking VIP configuration. 136 Checking VIP Subnet configuration. 137 Check for VIP Subnet configuration passed. 138 Checking VIP reachability 139 Check for VIP reachability passed. 140 141 Pre-check for node addition was successful. 142 Starting Oracle Universal Installer... 143 144 Checking swap space: must be greater than 500 MB. Actual 6143 MB Passed 145 Oracle Universal Installer, Version 11.2.0.4.0 Production 146 Copyright (C) 1999, 2013, Oracle. All rights reserved. 147 148 Performing tests to see whether nodes vastdata3 are available 149 ............................................................... 100% Done. 150 . 151 ----------------------------------------------------------------------------- 152 Cluster Node Addition Summary 153 Global Settings 154 Source: /u01/app/11.2.0/grid 155 New Nodes 156 Space Requirements 157 New Nodes 158 vastdata3 159 /: Required 4.61GB : Available 14.62GB 160 Installed Products 161 Product Names 162 Oracle Grid Infrastructure 11g 11.2.0.4.0 163 Java Development Kit 1.5.0.51.10 164 Installer SDK Component 11.2.0.4.0 165 Oracle One-Off Patch Installer 11.2.0.3.4 166 Oracle Universal Installer 11.2.0.4.0 167 Oracle RAC Required Support Files-HAS 11.2.0.4.0 168 Oracle USM Deconfiguration 11.2.0.4.0 169 Oracle Configuration Manager Deconfiguration 10.3.1.0.0 170 Enterprise Manager Common Core Files 10.2.0.4.5 171 Oracle DBCA Deconfiguration 11.2.0.4.0 172 Oracle RAC Deconfiguration 11.2.0.4.0 173 Oracle Quality of Service Management (Server) 11.2.0.4.0 174 Installation Plugin Files 11.2.0.4.0 175 Universal Storage Manager Files 11.2.0.4.0 176 Oracle Text Required Support Files 11.2.0.4.0 177 Automatic Storage Management Assistant 11.2.0.4.0 178 Oracle Database 11g Multimedia Files 11.2.0.4.0 179 Oracle Multimedia Java Advanced Imaging 11.2.0.4.0 180 Oracle Globalization Support 11.2.0.4.0 181 Oracle Multimedia Locator RDBMS Files 11.2.0.4.0 182 Oracle Core Required Support Files 11.2.0.4.0 183 Bali Share 1.1.18.0.0 184 Oracle Database Deconfiguration 11.2.0.4.0 185 Oracle Quality of Service Management (Client) 11.2.0.4.0 186 Expat libraries 2.0.1.0.1 187 Oracle Containers for Java 11.2.0.4.0 188 Perl Modules 5.10.0.0.1 189 Secure Socket Layer 11.2.0.4.0 190 Oracle JDBC/OCI Instant Client 11.2.0.4.0 191 Oracle Multimedia Client Option 11.2.0.4.0 192 LDAP Required Support Files 11.2.0.4.0 193 Character Set Migration Utility 11.2.0.4.0 194 Perl Interpreter 5.10.0.0.2 195 PL/SQL Embedded Gateway 11.2.0.4.0 196 OLAP SQL Scripts 11.2.0.4.0 197 Database SQL Scripts 11.2.0.4.0 198 Oracle Extended Windowing Toolkit 3.4.47.0.0 199 SSL Required Support Files for InstantClient 11.2.0.4.0 200 SQL*Plus Files for Instant Client 11.2.0.4.0 201 Oracle Net Required Support Files 11.2.0.4.0 202 Oracle Database User Interface 2.2.13.0.0 203 RDBMS Required Support Files for Instant Client 11.2.0.4.0 204 RDBMS Required Support Files Runtime 11.2.0.4.0 205 XML Parser for Java 11.2.0.4.0 206 Oracle Security Developer Tools 11.2.0.4.0 207 Oracle Wallet Manager 11.2.0.4.0 208 Enterprise Manager plugin Common Files 11.2.0.4.0 209 Platform Required Support Files 11.2.0.4.0 210 Oracle JFC Extended Windowing Toolkit 4.2.36.0.0 211 RDBMS Required Support Files 11.2.0.4.0 212 Oracle Ice Browser 5.2.3.6.0 213 Oracle Help For Java 4.2.9.0.0 214 Enterprise Manager Common Files 10.2.0.4.5 215 Deinstallation Tool 11.2.0.4.0 216 Oracle Java Client 11.2.0.4.0 217 Cluster Verification Utility Files 11.2.0.4.0 218 Oracle Notification Service (eONS) 11.2.0.4.0 219 Oracle LDAP administration 11.2.0.4.0 220 Cluster Verification Utility Common Files 11.2.0.4.0 221 Oracle Clusterware RDBMS Files 11.2.0.4.0 222 Oracle Locale Builder 11.2.0.4.0 223 Oracle Globalization Support 11.2.0.4.0 224 Buildtools Common Files 11.2.0.4.0 225 HAS Common Files 11.2.0.4.0 226 SQL*Plus Required Support Files 11.2.0.4.0 227 XDK Required Support Files 11.2.0.4.0 228 Agent Required Support Files 10.2.0.4.5 229 Parser Generator Required Support Files 11.2.0.4.0 230 Precompiler Required Support Files 11.2.0.4.0 231 Installation Common Files 11.2.0.4.0 232 Required Support Files 11.2.0.4.0 233 Oracle JDBC/THIN Interfaces 11.2.0.4.0 234 Oracle Multimedia Locator 11.2.0.4.0 235 Oracle Multimedia 11.2.0.4.0 236 Assistant Common Files 11.2.0.4.0 237 Oracle Net 11.2.0.4.0 238 PL/SQL 11.2.0.4.0 239 HAS Files for DB 11.2.0.4.0 240 Oracle Recovery Manager 11.2.0.4.0 241 Oracle Database Utilities 11.2.0.4.0 242 Oracle Notification Service 11.2.0.3.0 243 SQL*Plus 11.2.0.4.0 244 Oracle Netca Client 11.2.0.4.0 245 Oracle Advanced Security 11.2.0.4.0 246 Oracle JVM 11.2.0.4.0 247 Oracle Internet Directory Client 11.2.0.4.0 248 Oracle Net Listener 11.2.0.4.0 249 Cluster Ready Services Files 11.2.0.4.0 250 Oracle Database 11g 11.2.0.4.0 251 ----------------------------------------------------------------------------- 252 253 Instantiating scripts for add node (Monday, February 25, 2019 1:00:48 AM EST) 254 . 1% Done. 255 Instantiation of add node scripts complete 256 257 Copying to remote nodes (Monday, February 25, 2019 1:00:54 AM EST) 258 ............................................................................................... 96% Done. 259 Home copied to new nodes 260 261 Saving inventory on nodes (Monday, February 25, 2019 1:04:42 AM EST) 262 . 100% Done. 263 Save inventory complete 264 WARNING:A new inventory has been created on one or more nodes in this session. However, it has not yet been registered as the central inventory of this system. 265 To register the new inventory please run the script at '/u01/app/oraInventory/orainstRoot.sh' with root privileges on nodes 'vastdata3'. 266 If you do not register the inventory, you may not be able to update or patch the products you installed. 267 The following configuration scripts need to be executed as the "root" user in each new cluster node. Each script in the list below is followed by a list of nodes. 268 /u01/app/oraInventory/orainstRoot.sh #On nodes vastdata3 269 /u01/app/11.2.0/grid/root.sh #On nodes vastdata3 270 To execute the configuration scripts: 271 1. Open a terminal window 272 2. Log in as "root" 273 3. Run the scripts in each cluster node 274 275 The Cluster Node Addition of /u01/app/11.2.0/grid was successful. 276 Please check '/tmp/silentInstall.log' for more details.
[root@vastdata4 ~]# cat /tmp/silentInstall.log silentInstall.log sNativeVolName:/u01/app/11.2.0/grid/install/ m_asNodeArray:vastdata4,vastdata3 m_sLocalNode:vastdata4 WARNING: A new inventory has been created on one or more nodes in this session. However, it has not yet been registered as the central inventory of this system. To register the new inventory please run the script at '/u01/app/oraInventory/orainstRoot.sh' with root privileges on nodes 'vastdata3'. If you do not register the inventory, you may not be able to update or patch the products you installed. The following configuration scripts need to be executed as the "root" user in each new cluster node. Each script in the list below is followed by a list of nodes. /u01/app/oraInventory/orainstRoot.sh #On nodes vastdata3 /u01/app/11.2.0/grid/root.sh #On nodes vastdata3 To execute the configuration scripts: 1. Open a terminal window 2. Log in as "root" 3. Run the scripts in each cluster node The Cluster Node Addition of /u01/app/11.2.0/grid was successful.
[root@vastdata3 ~]# /u01/app/oraInventory/orainstRoot.sh Creating the Oracle inventory pointer file (/etc/oraInst.loc) Changing permissions of /u01/app/oraInventory. Adding read,write permissions for group. Removing read,write,execute permissions for world. Changing groupname of /u01/app/oraInventory to oinstall. The execution of the script is complete. [root@vastdata3 ~]# /u01/app/11.2.0/grid/root.sh Performing root user operation for Oracle 11g The following environment variables are set as: ORACLE_OWNER= grid ORACLE_HOME= /u01/app/11.2.0/grid Enter the full pathname of the local bin directory: [/usr/local/bin]: The contents of "dbhome" have not changed. No need to overwrite. The contents of "oraenv" have not changed. No need to overwrite. The contents of "coraenv" have not changed. No need to overwrite. Entries will be added to the /etc/oratab file as needed by Database Configuration Assistant when a database is created Finished running generic part of root script. Now product-specific root actions will be performed. Using configuration parameter file: /u01/app/11.2.0/grid/crs/install/crsconfig_params User ignored Prerequisites during installation Installing Trace File Analyzer OLR initialization - successful Adding Clusterware entries to upstart CRS-4402: The CSS daemon was started in exclusive mode but found an active CSS daemon on node vastdata4, number 2, and is terminating An active cluster was found during exclusive startup, restarting to join the cluster clscfg: EXISTING configuration version 5 detected. clscfg: version 5 is 11g Release 2. Successfully accumulated necessary OCR keys. Creating OCR keys for user 'root', privgrp 'root'.. Operation successful. Preparing packages for installation... cvuqdisk-1.0.9-1 Configure Oracle Grid Infrastructure for a Cluster ... succeeded
2.4节点添加后的检查
[grid@vastdata4 ~]$ cluvfy stage -post nodeadd -n vastdata3 -verbose
1 [grid@vastdata4 ~]$ cluvfy stage -post nodeadd -n vastdata3 -verbose 2 3 Performing post-checks for node addition 4 5 Checking node reachability... 6 7 Check: Node reachability from node "vastdata4" 8 Destination Node Reachable? 9 ------------------------------------ ------------------------ 10 vastdata3 yes 11 Result: Node reachability check passed from node "vastdata4" 12 13 14 Checking user equivalence... 15 16 Check: User equivalence for user "grid" 17 Node Name Status 18 ------------------------------------ ------------------------ 19 vastdata3 passed 20 Result: User equivalence check passed for user "grid" 21 22 Checking cluster integrity... 23 24 Node Name 25 ------------------------------------ 26 vastdata3 27 vastdata4 28 29 Cluster integrity check passed 30 31 32 Checking CRS integrity... 33 34 Clusterware version consistency passed 35 The Oracle Clusterware is healthy on node "vastdata4" 36 The Oracle Clusterware is healthy on node "vastdata3" 37 38 CRS integrity check passed 39 40 Checking shared resources... 41 42 Checking CRS home location... 43 "/u01/app/11.2.0/grid" is not shared 44 Result: Shared resources check for node addition passed 45 46 47 Checking node connectivity... 48 49 Checking hosts config file... 50 Node Name Status 51 ------------------------------------ ------------------------ 52 vastdata4 passed 53 vastdata3 passed 54 55 Verification of the hosts config file successful 56 57 58 Interface information for node "vastdata4" 59 Name IP Address Subnet Gateway Def. Gateway HW Address MTU 60 ------ --------------- --------------- --------------- --------------- ----------------- ------ 61 eth0 192.168.0.12 192.168.0.0 0.0.0.0 UNKNOWN 08:00:27:EF:F6:15 1500 62 eth0 192.168.0.33 192.168.0.0 0.0.0.0 UNKNOWN 08:00:27:EF:F6:15 1500 63 eth0 192.168.0.35 192.168.0.0 0.0.0.0 UNKNOWN 08:00:27:EF:F6:15 1500 64 eth0 192.168.0.23 192.168.0.0 0.0.0.0 UNKNOWN 08:00:27:EF:F6:15 1500 65 eth1 10.0.0.4 10.0.0.0 0.0.0.0 UNKNOWN 08:00:27:82:3A:57 1500 66 eth1 169.254.17.92 169.254.0.0 0.0.0.0 UNKNOWN 08:00:27:82:3A:57 1500 67 eth2 10.0.0.5 10.0.0.0 0.0.0.0 UNKNOWN 08:00:27:4C:5D:39 1500 68 eth2 169.254.236.167 169.254.128.0 0.0.0.0 UNKNOWN 08:00:27:4C:5D:39 1500 69 70 71 Interface information for node "vastdata3" 72 Name IP Address Subnet Gateway Def. Gateway HW Address MTU 73 ------ --------------- --------------- --------------- --------------- ----------------- ------ 74 eth0 192.168.0.11 192.168.0.0 0.0.0.0 UNKNOWN 08:00:27:D3:55:CB 1500 75 eth0 192.168.0.34 192.168.0.0 0.0.0.0 UNKNOWN 08:00:27:D3:55:CB 1500 76 eth0 192.168.0.22 192.168.0.0 0.0.0.0 UNKNOWN 08:00:27:D3:55:CB 1500 77 eth1 10.0.0.2 10.0.0.0 0.0.0.0 UNKNOWN 08:00:27:B4:C3:C7 1500 78 eth1 169.254.73.97 169.254.0.0 0.0.0.0 UNKNOWN 08:00:27:B4:C3:C7 1500 79 eth2 10.0.0.3 10.0.0.0 0.0.0.0 UNKNOWN 08:00:27:02:BD:9A 1500 80 eth2 169.254.222.114 169.254.128.0 0.0.0.0 UNKNOWN 08:00:27:02:BD:9A 1500 81 82 83 Check: Node connectivity for interface "eth0" 84 Source Destination Connected? 85 ------------------------------ ------------------------------ ---------------- 86 vastdata4[192.168.0.12] vastdata4[192.168.0.33] yes 87 vastdata4[192.168.0.12] vastdata4[192.168.0.35] yes 88 vastdata4[192.168.0.12] vastdata4[192.168.0.23] yes 89 vastdata4[192.168.0.12] vastdata3[192.168.0.11] yes 90 vastdata4[192.168.0.12] vastdata3[192.168.0.34] yes 91 vastdata4[192.168.0.12] vastdata3[192.168.0.22] yes 92 vastdata4[192.168.0.33] vastdata4[192.168.0.35] yes 93 vastdata4[192.168.0.33] vastdata4[192.168.0.23] yes 94 vastdata4[192.168.0.33] vastdata3[192.168.0.11] yes 95 vastdata4[192.168.0.33] vastdata3[192.168.0.34] yes 96 vastdata4[192.168.0.33] vastdata3[192.168.0.22] yes 97 vastdata4[192.168.0.35] vastdata4[192.168.0.23] yes 98 vastdata4[192.168.0.35] vastdata3[192.168.0.11] yes 99 vastdata4[192.168.0.35] vastdata3[192.168.0.34] yes 100 vastdata4[192.168.0.35] vastdata3[192.168.0.22] yes 101 vastdata4[192.168.0.23] vastdata3[192.168.0.11] yes 102 vastdata4[192.168.0.23] vastdata3[192.168.0.34] yes 103 vastdata4[192.168.0.23] vastdata3[192.168.0.22] yes 104 vastdata3[192.168.0.11] vastdata3[192.168.0.34] yes 105 vastdata3[192.168.0.11] vastdata3[192.168.0.22] yes 106 vastdata3[192.168.0.34] vastdata3[192.168.0.22] yes 107 Result: Node connectivity passed for interface "eth0" 108 109 110 Check: TCP connectivity of subnet "192.168.0.0" 111 Source Destination Connected? 112 ------------------------------ ------------------------------ ---------------- 113 vastdata4:192.168.0.12 vastdata4:192.168.0.33 passed 114 vastdata4:192.168.0.12 vastdata4:192.168.0.35 passed 115 vastdata4:192.168.0.12 vastdata4:192.168.0.23 passed 116 vastdata4:192.168.0.12 vastdata3:192.168.0.11 passed 117 vastdata4:192.168.0.12 vastdata3:192.168.0.34 passed 118 vastdata4:192.168.0.12 vastdata3:192.168.0.22 passed 119 Result: TCP connectivity check passed for subnet "192.168.0.0" 120 121 122 Check: Node connectivity for interface "eth1" 123 Source Destination Connected? 124 ------------------------------ ------------------------------ ---------------- 125 vastdata4[10.0.0.4] vastdata4[10.0.0.5] yes 126 vastdata4[10.0.0.4] vastdata3[10.0.0.2] yes 127 vastdata4[10.0.0.4] vastdata3[10.0.0.3] yes 128 vastdata4[10.0.0.5] vastdata3[10.0.0.2] yes 129 vastdata4[10.0.0.5] vastdata3[10.0.0.3] yes 130 vastdata3[10.0.0.2] vastdata3[10.0.0.3] yes 131 Result: Node connectivity passed for interface "eth1" 132 133 134 Check: TCP connectivity of subnet "10.0.0.0" 135 Source Destination Connected? 136 ------------------------------ ------------------------------ ---------------- 137 vastdata4:10.0.0.4 vastdata4:10.0.0.5 passed 138 vastdata4:10.0.0.4 vastdata3:10.0.0.2 passed 139 vastdata4:10.0.0.4 vastdata3:10.0.0.3 passed 140 Result: TCP connectivity check passed for subnet "10.0.0.0" 141 142 143 Check: Node connectivity for interface "eth2" 144 Checking subnet mask consistency... 145 Subnet mask consistency check passed for subnet "192.168.0.0". 146 Subnet mask consistency check passed for subnet "10.0.0.0". 147 Subnet mask consistency check passed. 148 149 Result: Node connectivity check passed 150 151 Checking multicast communication... 152 153 Checking subnet "192.168.0.0" for multicast communication with multicast group "230.0.1.0"... 154 Check of subnet "192.168.0.0" for multicast communication with multicast group "230.0.1.0" passed. 155 156 Checking subnet "10.0.0.0" for multicast communication with multicast group "230.0.1.0"... 157 Check of subnet "10.0.0.0" for multicast communication with multicast group "230.0.1.0" passed. 158 159 Check of multicast communication passed. 160 161 Checking node application existence... 162 163 Checking existence of VIP node application (required) 164 Node Name Required Running? Comment 165 ------------ ------------------------ ------------------------ ---------- 166 vastdata4 yes yes passed 167 vastdata3 yes yes passed 168 VIP node application check passed 169 170 Checking existence of NETWORK node application (required) 171 Node Name Required Running? Comment 172 ------------ ------------------------ ------------------------ ---------- 173 vastdata4 yes yes passed 174 vastdata3 yes yes passed 175 NETWORK node application check passed 176 177 Checking existence of GSD node application (optional) 178 Node Name Required Running? Comment 179 ------------ ------------------------ ------------------------ ---------- 180 vastdata4 no no exists 181 vastdata3 no no exists 182 GSD node application is offline on nodes "vastdata4,vastdata3" 183 184 Checking existence of ONS node application (optional) 185 Node Name Required Running? Comment 186 ------------ ------------------------ ------------------------ ---------- 187 vastdata4 no yes passed 188 vastdata3 no yes passed 189 ONS node application check passed 190 191 192 Checking Single Client Access Name (SCAN)... 193 SCAN Name Node Running? ListenerName Port Running? 194 ---------------- ------------ ------------ ------------ ------------ ------------ 195 vastdata-scan.us.oracle.com vastdata3 true LISTENER_SCAN1 1521 true 196 vastdata-scan.us.oracle.com vastdata4 true LISTENER_SCAN2 1521 true 197 vastdata-scan.us.oracle.com vastdata4 true LISTENER_SCAN3 1521 true 198 199 Checking TCP connectivity to SCAN Listeners... 200 Node ListenerName TCP connectivity? 201 ------------ ------------------------ ------------------------ 202 vastdata4 LISTENER_SCAN1 yes 203 vastdata4 LISTENER_SCAN2 yes 204 vastdata4 LISTENER_SCAN3 yes 205 TCP connectivity to SCAN Listeners exists on all cluster nodes 206 207 Checking name resolution setup for "vastdata-scan.us.oracle.com"... 208 209 Checking integrity of name service switch configuration file "/etc/nsswitch.conf" ... 210 Checking if "hosts" entry in file "/etc/nsswitch.conf" is consistent across nodes... 211 Checking file "/etc/nsswitch.conf" to make sure that only one "hosts" entry is defined 212 More than one "hosts" entry does not exist in any "/etc/nsswitch.conf" file 213 Check for integrity of name service switch configuration file "/etc/nsswitch.conf" passed 214 215 SCAN Name IP Address Status Comment 216 ------------ ------------------------ ------------------------ ---------- 217 vastdata-scan.us.oracle.com 192.168.0.33 passed 218 vastdata-scan.us.oracle.com 192.168.0.34 passed 219 vastdata-scan.us.oracle.com 192.168.0.35 passed 220 221 Verification of SCAN VIP and Listener setup passed 222 223 Checking to make sure user "grid" is not in "root" group 224 Node Name Status Comment 225 ------------ ------------------------ ------------------------ 226 vastdata3 passed does not exist 227 Result: User "grid" is not part of "root" group. Check passed 228 229 Checking if Clusterware is installed on all nodes... 230 Check of Clusterware install passed 231 232 Checking if CTSS Resource is running on all nodes... 233 Check: CTSS Resource running on all nodes 234 Node Name Status 235 ------------------------------------ ------------------------ 236 vastdata3 passed 237 Result: CTSS resource check passed 238 239 240 Querying CTSS for time offset on all nodes... 241 Result: Query of CTSS for time offset passed 242 243 Check CTSS state started... 244 Check: CTSS state 245 Node Name State 246 ------------------------------------ ------------------------ 247 vastdata3 Active 248 CTSS is in Active state. Proceeding with check of clock time offsets on all nodes... 249 Reference Time Offset Limit: 1000.0 msecs 250 Check: Reference Time Offset 251 Node Name Time Offset Status 252 ------------ ------------------------ ------------------------ 253 vastdata3 0.0 passed 254 255 Time offset is within the specified limits on the following set of nodes: 256 "[vastdata3]" 257 Result: Check of clock time offsets passed 258 259 Oracle Cluster Time Synchronization Services check passed 260 261 Post-check for node addition was successful.
2.5检查集群内节点状态
[grid@vastdata4 ~]$ olsnodes -t vastdata3 Unpinned vastdata4 Unpinned
2.6检查集群资源状态
[grid@vastdata4 ~]$ crsctl stat res -t -------------------------------------------------------------------------------- NAME TARGET STATE SERVER STATE_DETAILS -------------------------------------------------------------------------------- Local Resources -------------------------------------------------------------------------------- ora.DATA.dg ONLINE ONLINE vastdata3 ONLINE ONLINE vastdata4 ora.FRA.dg ONLINE ONLINE vastdata3 ONLINE ONLINE vastdata4 ora.LISTENER.lsnr ONLINE ONLINE vastdata3 ONLINE ONLINE vastdata4 ora.asm ONLINE ONLINE vastdata3 Started ONLINE ONLINE vastdata4 Started ora.gsd OFFLINE OFFLINE vastdata3 OFFLINE OFFLINE vastdata4 ora.net1.network ONLINE ONLINE vastdata3 ONLINE ONLINE vastdata4 ora.ons ONLINE ONLINE vastdata3 ONLINE ONLINE vastdata4 ora.registry.acfs ONLINE ONLINE vastdata3 ONLINE ONLINE vastdata4 -------------------------------------------------------------------------------- Cluster Resources -------------------------------------------------------------------------------- ora.LISTENER_SCAN1.lsnr 1 ONLINE ONLINE vastdata3 ora.LISTENER_SCAN2.lsnr 1 ONLINE ONLINE vastdata4 ora.LISTENER_SCAN3.lsnr 1 ONLINE ONLINE vastdata4 ora.cvu 1 ONLINE ONLINE vastdata4 ora.oc4j 1 ONLINE ONLINE vastdata4 ora.prod.db 2 ONLINE ONLINE vastdata4 Open ora.scan1.vip 1 ONLINE ONLINE vastdata3 ora.scan2.vip 1 ONLINE ONLINE vastdata4 ora.scan3.vip 1 ONLINE ONLINE vastdata4 ora.vastdata3.vip 1 ONLINE ONLINE vastdata3 ora.vastdata4.vip 1 ONLINE ONLINE vastdata4
3.新节点安装 DB 软件
[oracle@vastdata4 ~]$ $ORACLE_HOME/oui/bin/addNode.sh -silent "CLUSTER_NEW_NODES={vastdata3}"
1 [oracle@vastdata4 ~]$ $ORACLE_HOME/oui/bin/addNode.sh -silent "CLUSTER_NEW_NODES={vastdata3}" 2 3 Performing pre-checks for node addition 4 5 Checking node reachability... 6 Node reachability check passed from node "vastdata4" 7 8 Checking user equivalence... 9 User equivalence check passed for user "oracle" 10 11 WARNING: 12 Node "vastdata3" already appears to be part of cluster 13 14 Pre-check for node addition was successful. 15 Starting Oracle Universal Installer... 16 17 Checking swap space: must be greater than 500 MB. Actual 6143 MB Passed 18 Oracle Universal Installer, Version 11.2.0.4.0 Production 19 Copyright (C) 1999, 2013, Oracle. All rights reserved. 20 21 22 Performing tests to see whether nodes vastdata3 are available 23 ............................................................... 100% Done. 24 25 .. 26 ----------------------------------------------------------------------------- 27 Cluster Node Addition Summary 28 Global Settings 29 Source: /u01/app/oracle/product/11.2.0/db_1 30 New Nodes 31 Space Requirements 32 New Nodes 33 vastdata3 34 /: Required 4.34GB : Available 10.59GB 35 Installed Products 36 Product Names 37 Oracle Database 11g 11.2.0.4.0 38 Java Development Kit 1.5.0.51.10 39 Installer SDK Component 11.2.0.4.0 40 Oracle One-Off Patch Installer 11.2.0.3.4 41 Oracle Universal Installer 11.2.0.4.0 42 Oracle USM Deconfiguration 11.2.0.4.0 43 Oracle Configuration Manager Deconfiguration 10.3.1.0.0 44 Oracle DBCA Deconfiguration 11.2.0.4.0 45 Oracle RAC Deconfiguration 11.2.0.4.0 46 Oracle Database Deconfiguration 11.2.0.4.0 47 Oracle Configuration Manager Client 10.3.2.1.0 48 Oracle Configuration Manager 10.3.8.1.0 49 Oracle ODBC Driverfor Instant Client 11.2.0.4.0 50 LDAP Required Support Files 11.2.0.4.0 51 SSL Required Support Files for InstantClient 11.2.0.4.0 52 Bali Share 1.1.18.0.0 53 Oracle Extended Windowing Toolkit 3.4.47.0.0 54 Oracle JFC Extended Windowing Toolkit 4.2.36.0.0 55 Oracle Real Application Testing 11.2.0.4.0 56 Oracle Database Vault J2EE Application 11.2.0.4.0 57 Oracle Label Security 11.2.0.4.0 58 Oracle Data Mining RDBMS Files 11.2.0.4.0 59 Oracle OLAP RDBMS Files 11.2.0.4.0 60 Oracle OLAP API 11.2.0.4.0 61 Platform Required Support Files 11.2.0.4.0 62 Oracle Database Vault option 11.2.0.4.0 63 Oracle RAC Required Support Files-HAS 11.2.0.4.0 64 SQL*Plus Required Support Files 11.2.0.4.0 65 Oracle Display Fonts 9.0.2.0.0 66 Oracle Ice Browser 5.2.3.6.0 67 Oracle JDBC Server Support Package 11.2.0.4.0 68 Oracle SQL Developer 11.2.0.4.0 69 Oracle Application Express 11.2.0.4.0 70 XDK Required Support Files 11.2.0.4.0 71 RDBMS Required Support Files for Instant Client 11.2.0.4.0 72 SQLJ Runtime 11.2.0.4.0 73 Database Workspace Manager 11.2.0.4.0 74 RDBMS Required Support Files Runtime 11.2.0.4.0 75 Oracle Globalization Support 11.2.0.4.0 76 Exadata Storage Server 11.2.0.1.0 77 Provisioning Advisor Framework 10.2.0.4.3 78 Enterprise Manager Database Plugin -- Repository Support 11.2.0.4.0 79 Enterprise Manager Repository Core Files 10.2.0.4.5 80 Enterprise Manager Database Plugin -- Agent Support 11.2.0.4.0 81 Enterprise Manager Grid Control Core Files 10.2.0.4.5 82 Enterprise Manager Common Core Files 10.2.0.4.5 83 Enterprise Manager Agent Core Files 10.2.0.4.5 84 RDBMS Required Support Files 11.2.0.4.0 85 regexp 2.1.9.0.0 86 Agent Required Support Files 10.2.0.4.5 87 Oracle 11g Warehouse Builder Required Files 11.2.0.4.0 88 Oracle Notification Service (eONS) 11.2.0.4.0 89 Oracle Text Required Support Files 11.2.0.4.0 90 Parser Generator Required Support Files 11.2.0.4.0 91 Oracle Database 11g Multimedia Files 11.2.0.4.0 92 Oracle Multimedia Java Advanced Imaging 11.2.0.4.0 93 Oracle Multimedia Annotator 11.2.0.4.0 94 Oracle JDBC/OCI Instant Client 11.2.0.4.0 95 Oracle Multimedia Locator RDBMS Files 11.2.0.4.0 96 Precompiler Required Support Files 11.2.0.4.0 97 Oracle Core Required Support Files 11.2.0.4.0 98 Sample Schema Data 11.2.0.4.0 99 Oracle Starter Database 11.2.0.4.0 100 Oracle Message Gateway Common Files 11.2.0.4.0 101 Oracle XML Query 11.2.0.4.0 102 XML Parser for Oracle JVM 11.2.0.4.0 103 Oracle Help For Java 4.2.9.0.0 104 Installation Plugin Files 11.2.0.4.0 105 Enterprise Manager Common Files 10.2.0.4.5 106 Expat libraries 2.0.1.0.1 107 Deinstallation Tool 11.2.0.4.0 108 Oracle Quality of Service Management (Client) 11.2.0.4.0 109 Perl Modules 5.10.0.0.1 110 JAccelerator (COMPANION) 11.2.0.4.0 111 Oracle Containers for Java 11.2.0.4.0 112 Perl Interpreter 5.10.0.0.2 113 Oracle Net Required Support Files 11.2.0.4.0 114 Secure Socket Layer 11.2.0.4.0 115 Oracle Universal Connection Pool 11.2.0.4.0 116 Oracle JDBC/THIN Interfaces 11.2.0.4.0 117 Oracle Multimedia Client Option 11.2.0.4.0 118 Oracle Java Client 11.2.0.4.0 119 Character Set Migration Utility 11.2.0.4.0 120 Oracle Code Editor 1.2.1.0.0I 121 PL/SQL Embedded Gateway 11.2.0.4.0 122 OLAP SQL Scripts 11.2.0.4.0 123 Database SQL Scripts 11.2.0.4.0 124 Oracle Locale Builder 11.2.0.4.0 125 Oracle Globalization Support 11.2.0.4.0 126 SQL*Plus Files for Instant Client 11.2.0.4.0 127 Required Support Files 11.2.0.4.0 128 Oracle Database User Interface 2.2.13.0.0 129 Oracle ODBC Driver 11.2.0.4.0 130 Oracle Notification Service 11.2.0.3.0 131 XML Parser for Java 11.2.0.4.0 132 Oracle Security Developer Tools 11.2.0.4.0 133 Oracle Wallet Manager 11.2.0.4.0 134 Cluster Verification Utility Common Files 11.2.0.4.0 135 Oracle Clusterware RDBMS Files 11.2.0.4.0 136 Oracle UIX 2.2.24.6.0 137 Enterprise Manager plugin Common Files 11.2.0.4.0 138 HAS Common Files 11.2.0.4.0 139 Precompiler Common Files 11.2.0.4.0 140 Installation Common Files 11.2.0.4.0 141 Oracle Help for the Web 2.0.14.0.0 142 Oracle LDAP administration 11.2.0.4.0 143 Buildtools Common Files 11.2.0.4.0 144 Assistant Common Files 11.2.0.4.0 145 Oracle Recovery Manager 11.2.0.4.0 146 PL/SQL 11.2.0.4.0 147 Generic Connectivity Common Files 11.2.0.4.0 148 Oracle Database Gateway for ODBC 11.2.0.4.0 149 Oracle Programmer 11.2.0.4.0 150 Oracle Database Utilities 11.2.0.4.0 151 Enterprise Manager Agent 10.2.0.4.5 152 SQL*Plus 11.2.0.4.0 153 Oracle Netca Client 11.2.0.4.0 154 Oracle Multimedia Locator 11.2.0.4.0 155 Oracle Call Interface (OCI) 11.2.0.4.0 156 Oracle Multimedia 11.2.0.4.0 157 Oracle Net 11.2.0.4.0 158 Oracle XML Development Kit 11.2.0.4.0 159 Oracle Internet Directory Client 11.2.0.4.0 160 Database Configuration and Upgrade Assistants 11.2.0.4.0 161 Oracle JVM 11.2.0.4.0 162 Oracle Advanced Security 11.2.0.4.0 163 Oracle Net Listener 11.2.0.4.0 164 Oracle Enterprise Manager Console DB 11.2.0.4.0 165 HAS Files for DB 11.2.0.4.0 166 Oracle Text 11.2.0.4.0 167 Oracle Net Services 11.2.0.4.0 168 Oracle Database 11g 11.2.0.4.0 169 Oracle OLAP 11.2.0.4.0 170 Oracle Spatial 11.2.0.4.0 171 Oracle Partitioning 11.2.0.4.0 172 Enterprise Edition Options 11.2.0.4.0 173 ----------------------------------------------------------------------------- 174 175 176 Instantiating scripts for add node (Monday, February 25, 2019 1:20:38 AM EST) 177 . 1% Done. 178 Instantiation of add node scripts complete 179 180 Copying to remote nodes (Monday, February 25, 2019 1:20:44 AM EST) 181 ............................................................................................... 96% Done. 182 Home copied to new nodes 183 184 Saving inventory on nodes (Monday, February 25, 2019 1:26:47 AM EST) 185 . 100% Done. 186 Save inventory complete 187 WARNING: 188 The following configuration scripts need to be executed as the "root" user in each new cluster node. Each script in the list below is followed by a list of nodes. 189 /u01/app/oracle/product/11.2.0/db_1/root.sh #On nodes vastdata3 190 To execute the configuration scripts: 191 1. Open a terminal window 192 2. Log in as "root" 193 3. Run the scripts in each cluster node 194 195 The Cluster Node Addition of /u01/app/oracle/product/11.2.0/db_1 was successful. 196 Please check '/tmp/silentInstall.log' for more details.
[oracle@vastdata4 ~]$ cat /tmp/silentInstall.log silentInstall.log sNativeVolName:/u01/app/11.2.0/grid/install/ m_asNodeArray:vastdata4,vastdata3 m_sLocalNode:vastdata4 WARNING: A new inventory has been created on one or more nodes in this session. However, it has not yet been registered as the central inventory of this system. To register the new inventory please run the script at '/u01/app/oraInventory/orainstRoot.sh' with root privileges on nodes 'vastdata3'. If you do not register the inventory, you may not be able to update or patch the products you installed. The following configuration scripts need to be executed as the "root" user in each new cluster node. Each script in the list below is followed by a list of nodes. /u01/app/oraInventory/orainstRoot.sh #On nodes vastdata3 /u01/app/11.2.0/grid/root.sh #On nodes vastdata3 To execute the configuration scripts: 1. Open a terminal window 2. Log in as "root" 3. Run the scripts in each cluster node The Cluster Node Addition of /u01/app/11.2.0/grid was successful.
[root@vastdata3 ~]# /u01/app/oraInventory/orainstRoot.sh Creating the Oracle inventory pointer file (/etc/oraInst.loc) Changing permissions of /u01/app/oraInventory. Adding read,write permissions for group. Removing read,write,execute permissions for world. Changing groupname of /u01/app/oraInventory to oinstall. The execution of the script is complete. [root@vastdata3 ~]# /u01/app/11.2.0/grid/root.sh Performing root user operation for Oracle 11g The following environment variables are set as: ORACLE_OWNER= grid ORACLE_HOME= /u01/app/11.2.0/grid Enter the full pathname of the local bin directory: [/usr/local/bin]: The contents of "dbhome" have not changed. No need to overwrite. The contents of "oraenv" have not changed. No need to overwrite. The contents of "coraenv" have not changed. No need to overwrite. Entries will be added to the /etc/oratab file as needed by Database Configuration Assistant when a database is created Finished running generic part of root script. Now product-specific root actions will be performed. Using configuration parameter file: /u01/app/11.2.0/grid/crs/install/crsconfig_params User ignored Prerequisites during installation Installing Trace File Analyzer Configure Oracle Grid Infrastructure for a Cluster ... succeeded
4.给新节点分配实例
4.1检查 RAC 下所有实例状态
[oracle@vastdata4 ~]$ srvctl status database -d PROD -f Instance PROD2 is running on node vastdata4 [oracle@vastdata4 ~]$ srvctl config database -d PROD -a Database unique name: PROD Database name: PROD Oracle home: /u01/app/oracle/product/11.2.0/db_1 Oracle user: oracle Spfile: +DATA/PROD/spfilePROD.ora Domain: Start options: open Stop options: immediate Database role: PRIMARY Management policy: AUTOMATIC Server pools: PROD Database instances: PROD2 Disk Groups: DATA,FRA Mount point paths: Services: Type: RAC Database is enabled Database is administrator managed
4.2为新增节点分配实例
[oracle@vastdata4 ~]$ dbca -silent -addInstance -gdbName PROD -nodelist vastdata3 -instanceName PROD1 -sysDBAUserName sys -sysDBAPassword oracle Adding instance 1% complete 2% complete 6% complete 13% complete 20% complete 26% complete 33% complete 40% complete 46% complete 53% complete 66% complete Completing instance management. 76% complete 100% complete Look at the log file "/u01/app/oracle/cfgtoollogs/dbca/PROD/PROD.log" for further details.
[oracle@vastdata4 ~]$ srvctl config database -d PROD -a Database unique name: PROD Database name: PROD Oracle home: /u01/app/oracle/product/11.2.0/db_1 Oracle user: oracle Spfile: +DATA/PROD/spfilePROD.ora Domain: Start options: open Stop options: immediate Database role: PRIMARY Management policy: AUTOMATIC Server pools: PROD Database instances: PROD1,PROD2 Disk Groups: DATA,FRA Mount point paths: Services: Type: RAC Database is enabled Database is administrator managed
4.3检查集群资源状态
[grid@vastdata4 ~]$ crsctl stat res -t -------------------------------------------------------------------------------- NAME TARGET STATE SERVER STATE_DETAILS -------------------------------------------------------------------------------- Local Resources -------------------------------------------------------------------------------- ora.DATA.dg ONLINE ONLINE vastdata3 ONLINE ONLINE vastdata4 ora.FRA.dg ONLINE ONLINE vastdata3 ONLINE ONLINE vastdata4 ora.LISTENER.lsnr ONLINE ONLINE vastdata3 ONLINE ONLINE vastdata4 ora.asm ONLINE ONLINE vastdata3 Started ONLINE ONLINE vastdata4 Started ora.gsd OFFLINE OFFLINE vastdata3 OFFLINE OFFLINE vastdata4 ora.net1.network ONLINE ONLINE vastdata3 ONLINE ONLINE vastdata4 ora.ons ONLINE ONLINE vastdata3 ONLINE ONLINE vastdata4 ora.registry.acfs ONLINE ONLINE vastdata3 ONLINE ONLINE vastdata4 -------------------------------------------------------------------------------- Cluster Resources -------------------------------------------------------------------------------- ora.LISTENER_SCAN1.lsnr 1 ONLINE ONLINE vastdata3 ora.LISTENER_SCAN2.lsnr 1 ONLINE ONLINE vastdata4 ora.LISTENER_SCAN3.lsnr 1 ONLINE ONLINE vastdata4 ora.cvu 1 ONLINE ONLINE vastdata4 ora.oc4j 1 ONLINE ONLINE vastdata4 ora.prod.db 1 ONLINE ONLINE vastdata3 Open 2 ONLINE ONLINE vastdata4 Open ora.scan1.vip 1 ONLINE ONLINE vastdata3 ora.scan2.vip 1 ONLINE ONLINE vastdata4 ora.scan3.vip 1 ONLINE ONLINE vastdata4 ora.vastdata3.vip 1 ONLINE ONLINE vastdata3 ora.vastdata4.vip 1 ONLINE ONLINE vastdata4
4.4检查新添加节点数据库的运行情况
[oracle@vastdata3 ~]$ sqlplus / as sysdba SQL*Plus: Release 11.2.0.4.0 Production on Mon Feb 25 01:44:56 2019 Copyright (c) 1982, 2013, Oracle. All rights reserved. Connected to: Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP, Data Mining and Real Application Testing options SQL> select count(*) from tab; COUNT(*) ---------- 4987
至此，Oracle RAC架构搭建完成。
如有转载，请标明出处。