Oracle 11gR2 RAC 删除节点

1.  概述

1.1   目标

生产数据库删除节点。

2.  删除前准备

2.1   DBCA删除数据库实例

在节点1以oracle用户运行dbca

2.2   RAC层面删除节点

node1 操作停止listener

[grid@node1-11gr2 ~]$ srvctl disable listener -l LISTENER -n node2-11gr2

[grid@node1-11gr2 ~]$

[grid@node1-11gr2 ~]$ srvctl stop listener -l LISTENER -n node2-11gr2

在node1更新inventory

[oracle@node1-11gr2 bin]$ echo $ORACLE_HOME

/u01/app/oracle/product/11.2.0/db_1

[oracle@node1-11gr2 bin]$ ./runInstaller -updatenodelist ORACLE_HOME=/u01/app/oracle/product/11.2.0/db_1 "CLUSTER_NODES={node1-11gr2,node3-11gr2,node4-11gr2}"

Starting Oracle Universal Installer...

Checking swap space: must be greater than 500 MB.   Actual 7644 MB    Passed

The inventory pointer is located at /etc/oraInst.loc

The inventory is located at /u01/app/oraInventory

'UpdateNodeList' was successful.

 

[grid@node1-11gr2 ContentsXML]$ cat inventory.xml

<?xml version="1.0" standalone="yes" ?>

<!-- Copyright (c) 1999, 2013, Oracle and/or its affiliates.

All rights reserved. -->

<!-- Do not modify the contents of this file by hand. -->

<INVENTORY>

<VERSION_INFO>

   <SAVED_WITH>11.2.0.4.0</SAVED_WITH>

   <MINIMUM_VER>2.1.0.6.0</MINIMUM_VER>

</VERSION_INFO>

<HOME_LIST>

<HOME NAME="Ora11g_gridinfrahome1" LOC="/u01/11.2.0/grid" TYPE="O" IDX="1" CRS="true">

   <NODE_LIST>

      <NODE NAME="node1-11gr2"/>

      <NODE NAME="node2-11gr2"/>

      <NODE NAME="node3-11gr2"/>

      <NODE NAME="node4-11gr2"/>

   </NODE_LIST>

</HOME>

<HOME NAME="OraDb11g_home1" LOC="/u01/app/oracle/product/11.2.0/db_1" TYPE="O" IDX="2">

   <NODE_LIST>

      <NODE NAME="node1-11gr2"/>

      <NODE NAME="node2-11gr2"/>

      <NODE NAME="node3-11gr2"/>

      <NODE NAME="node4-11gr2"/>

   </NODE_LIST>

</HOME>

</HOME_LIST>

node1操作

[grid@node1-11gr2 ~]$ srvctl status database -d prod

Instance prod1 is running on node node1-11gr2

Instance prod3 is running on node node3-11gr2

Instance prod4 is running on node node4-11gr2

 删除ORACLE HOME，在节点2运行Deinstall

[oracle@node2-11gr2 deinstall]$ pwd

/u01/app/oracle/product/11.2.0/db_1/deinstall

[oracle@node2-11gr2 deinstall]$ pwd

/u01/app/oracle/product/11.2.0/db_1/deinstall

[oracle@node2-11gr2 deinstall]$

[oracle@node2-11gr2 deinstall]$

[oracle@node2-11gr2 deinstall]$ ./deinstall -local

Checking for required files and bootstrapping ...

Please wait ...

Location of logs /u01/app/oraInventory/logs/

 

############ ORACLE DEINSTALL & DECONFIG TOOL START ############

 

 

######################### CHECK OPERATION START #########################

## [START] Install check configuration ##

Checking for existence of the Oracle home location /u01/app/oracle/product/11.2.0/db_1

Oracle Home type selected for deinstall is: Oracle Real Application Cluster Database

Oracle Base selected for deinstall is: /u01/app/oracle

Checking for existence of central inventory location /u01/app/oraInventory

Checking for existence of the Oracle Grid Infrastructure home /u01/11.2.0/grid

The following nodes are part of this cluster: node1-11gr2,node2-11gr2,node3-11gr2,node4-11gr2

Checking for sufficient temp space availability on node(s) : 'node1-11gr2,node2-11gr2,node3-11gr2,node4-11gr2'

 

## [END] Install check configuration ##

 Network Configuration check config START

 Network de-configuration trace file location: /u01/app/oraInventory/logs/netdc_check2017-01-19_03-14-47-PM.log

 Network Configuration check config END

 Database Check Configuration START

 Database de-configuration trace file location: /u01/app/oraInventory/logs/databasedc_check2017-01-19_03-14-54-PM.log

 Database Check Configuration END

 Enterprise Manager Configuration Assistant START

 EMCA de-configuration trace file location: /u01/app/oraInventory/logs/emcadc_check2017-01-19_03-14-56-PM.log

 

Enterprise Manager Configuration Assistant END

Oracle Configuration Manager check START

OCM check log file location : /u01/app/oraInventory/logs//ocm_check3675.log

Oracle Configuration Manager check END

 

######################### CHECK OPERATION END #########################

 

####################### CHECK OPERATION SUMMARY #######################

Oracle Grid Infrastructure Home is: /u01/11.2.0/grid

The cluster node(s) on which the Oracle home deinstallation will be performed are:node1-11gr2,node2-11gr2,node3-11gr2,node4-11gr2

Since -local option has been specified, the Oracle home will be deinstalled only on the local node, 'node2-11gr2', and the global configuration will be removed.

Oracle Home selected for deinstall is: /u01/app/oracle/product/11.2.0/db_1

Inventory Location where the Oracle home registered is: /u01/app/oraInventory

The option -local will not modify any database configuration for this Oracle home.

 

No Enterprise Manager configuration to be updated for any database(s)

No Enterprise Manager ASM targets to update

No Enterprise Manager listener targets to migrate

Checking the config status for CCR

node1-11gr2 : Oracle Home exists with CCR directory, but CCR is not configured

node2-11gr2 : Oracle Home exists with CCR directory, but CCR is not configured

node3-11gr2 : Oracle Home exists with CCR directory, but CCR is not configured

node4-11gr2 : Oracle Home exists with CCR directory, but CCR is not configured

CCR check is finished

Do you want to continue (y - yes, n - no)? [n]: y 

A log of this session will be written to: '/u01/app/oraInventory/logs/deinstall_deconfig2017-01-19_03-14-39-PM.out'

Any error messages from this session will be written to: '/u01/app/oraInventory/logs/deinstall_deconfig2017-01-19_03-14-39-PM.err'

######################## CLEAN OPERATION START ########################

Enterprise Manager Configuration Assistant START

EMCA de-configuration trace file location: /u01/app/oraInventory/logs/emcadc_clean2017-01-19_03-14-56-PM.log

Updating Enterprise Manager ASM targets (if any)

Updating Enterprise Manager listener targets (if any)

Enterprise Manager Configuration Assistant END

Database de-configuration trace file location: /u01/app/oraInventory/logs/databasedc_clean2017-01-19_03-15-06-PM.log

Network Configuration clean config START

Network de-configuration trace file location: /u01/app/oraInventory/logs/netdc_clean2017-01-19_03-15-06-PM.log

De-configuring Local Net Service Names configuration file...

Local Net Service Names configuration file de-configured successfully.

De-configuring backup files...

Backup files de-configured successfully.

The network configuration has been cleaned up successfully.

Network Configuration clean config END

Oracle Configuration Manager clean START

OCM clean log file location : /u01/app/oraInventory/logs//ocm_clean3675.log

Oracle Configuration Manager clean END

Setting the force flag to false

Setting the force flag to cleanup the Oracle Base

Oracle Universal Installer clean START

Detach Oracle home '/u01/app/oracle/product/11.2.0/db_1' from the central inventory on the local node : Done

Delete directory '/u01/app/oracle/product/11.2.0/db_1' on the local node : Done

The Oracle Base directory '/u01/app/oracle' will not be removed on local node. The directory is in use by Oracle Home '/u01/11.2.0/grid'.

Oracle Universal Installer cleanup was successful.

Oracle Universal Installer clean END

## [START] Oracle install clean ##

Clean install operation removing temporary directory '/tmp/deinstall2017-01-19_03-13-39PM' on node 'node2-11gr2'

## [END] Oracle install clean ##

######################### CLEAN OPERATION END #########################

####################### CLEAN OPERATION SUMMARY #######################

Cleaning the config for CCR

As CCR is not configured, so skipping the cleaning of CCR configuration

CCR clean is finished

Successfully detached Oracle home '/u01/app/oracle/product/11.2.0/db_1' from the central inventory on the local node.

Successfully deleted directory '/u01/app/oracle/product/11.2.0/db_1' on the local node.

Oracle Universal Installer cleanup was successful.

Oracle deinstall tool successfully cleaned up temporary directories.

#######################################################################

############# ORACLE DEINSTALL & DECONFIG TOOL END #############

2.3   在GRID层面删除节点

[grid@node1-11gr2 ~]$  olsnodes -s -t

node1-11gr2     Active  Unpinned

node2-11gr2     Active  Unpinned

node3-11gr2     Active  Unpinned

node4-11gr2     Active  Unpinned

如果是pinned,请设为Unpinned
crsctl unpin css -n 被删除节点名

在被删除节点禁用clusterware的applications and daemons

su - root
# cd $GRID_HOME/crs/install
# ./rootcrs.pl -deconfig -force

在节点2以root用户运行deconfig

[root@node2-11gr2 install]# pwd

/u01/11.2.0/grid/crs/install

[root@node2-11gr2 install]#  ./rootcrs.pl -deconfig -deinstall -force

Using configuration parameter file: ./crsconfig_params

Network exists: 1/192.168.150.0/255.255.255.0/eth0, type static

VIP exists: /node1-11gr2-vip/192.168.150.133/192.168.150.0/255.255.255.0/eth0, hosting node node1-11gr2

VIP exists: /node2-11gr2-vip/192.168.150.134/192.168.150.0/255.255.255.0/eth0, hosting node node2-11gr2

VIP exists: /node3-11gr2-vip/192.168.150.136/192.168.150.0/255.255.255.0/eth0, hosting node node3-11gr2

VIP exists: /node4-11gr2-vip/192.168.150.137/192.168.150.0/255.255.255.0/eth0, hosting node node4-11gr2

GSD exists

ONS exists: Local port 6100, remote port 6200, EM port 2016

CRS-2673: Attempting to stop 'ora.registry.acfs' on 'node2-11gr2'

CRS-2677: Stop of 'ora.registry.acfs' on 'node2-11gr2' succeeded

CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'node2-11gr2'

CRS-2673: Attempting to stop 'ora.crsd' on 'node2-11gr2'

CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on 'node2-11gr2'

CRS-2673: Attempting to stop 'ora.CRS.dg' on 'node2-11gr2'

CRS-2673: Attempting to stop 'ora.DG1.dg' on 'node2-11gr2'

CRS-2677: Stop of 'ora.DG1.dg' on 'node2-11gr2' succeeded

CRS-2677: Stop of 'ora.CRS.dg' on 'node2-11gr2' succeeded

CRS-2673: Attempting to stop 'ora.asm' on 'node2-11gr2'

CRS-2677: Stop of 'ora.asm' on 'node2-11gr2' succeeded

CRS-2792: Shutdown of Cluster Ready Services-managed resources on 'node2-11gr2' has completed

CRS-2677: Stop of 'ora.crsd' on 'node2-11gr2' succeeded

CRS-2673: Attempting to stop 'ora.mdnsd' on 'node2-11gr2'

CRS-2673: Attempting to stop 'ora.ctssd' on 'node2-11gr2'

CRS-2673: Attempting to stop 'ora.evmd' on 'node2-11gr2'

CRS-2673: Attempting to stop 'ora.asm' on 'node2-11gr2'

CRS-2673: Attempting to stop 'ora.drivers.acfs' on 'node2-11gr2'

CRS-2677: Stop of 'ora.mdnsd' on 'node2-11gr2' succeeded

CRS-2677: Stop of 'ora.evmd' on 'node2-11gr2' succeeded

CRS-2677: Stop of 'ora.ctssd' on 'node2-11gr2' succeeded

CRS-2677: Stop of 'ora.drivers.acfs' on 'node2-11gr2' succeeded

CRS-2677: Stop of 'ora.asm' on 'node2-11gr2' succeeded

CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on 'node2-11gr2'

CRS-2677: Stop of 'ora.cluster_interconnect.haip' on 'node2-11gr2' succeeded

CRS-2673: Attempting to stop 'ora.cssd' on 'node2-11gr2'

CRS-2677: Stop of 'ora.cssd' on 'node2-11gr2' succeeded

CRS-2673: Attempting to stop 'ora.crf' on 'node2-11gr2'

CRS-2677: Stop of 'ora.crf' on 'node2-11gr2' succeeded

CRS-2673: Attempting to stop 'ora.gipcd' on 'node2-11gr2'

CRS-2677: Stop of 'ora.gipcd' on 'node2-11gr2' succeeded

CRS-2673: Attempting to stop 'ora.gpnpd' on 'node2-11gr2'

CRS-2677: Stop of 'ora.gpnpd' on 'node2-11gr2' succeeded

CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'node2-11gr2' has completed

CRS-4133: Oracle High Availability Services has been stopped.

Removing Trace File Analyzer

Successfully deconfigured Oracle clusterware stack on this node

[grid@node1-11gr2 ~]$  olsnodes -s -t

node1-11gr2     Active  Unpinned

node2-11gr2     Inactive        Unpinned

node3-11gr2     Active  Unpinned

node4-11gr2     Active  Unpinned

在节点1运行，删除节点将被删除节点从节点信息中删除

su - root
# crsctl delete node -n node_to_be_deleted

[root@node1-11gr2 grid]# crsctl delete node -n node2-11gr2

CRS-4661: Node node2-11gr2 successfully deleted

[root@node1-11gr2 grid]#  olsnodes -s -t

node1-11gr2     Active  Unpinned

node3-11gr2     Active  Unpinned

node4-11gr2     Active  Unpinned

在节点2更新节点列表

su - grid
cd $Grid_home/oui/bin
$ ./runInstaller -updateNodeList ORACLE_HOME=Grid_home "CLUSTER_NODES={remaining_nodes_list}" CRS=TRUE -silent

[grid@node2-11gr2 bin]$  ./runInstaller -updateNodeList ORACLE_HOME=/u01/11.2.0/grid  "CLUSTER_NODES={node2-11gr2}" CRS=TRUE  -local

Starting Oracle Universal Installer...

Checking swap space: must be greater than 500 MB.   Actual 8151 MB    Passed

The inventory pointer is located at /etc/oraInst.loc

The inventory is located at /u01/app/oraInventory

'UpdateNodeList' was successful.

卸载GI,删除grid home

被删除节点，节点2执行
su - grid
cd $Grid_home/deinstall
./deinstall –local

[grid@node2-11gr2 deinstall]$ pwd

/u01/11.2.0/grid/deinstall

[grid@node2-11gr2 deinstall]$

[grid@node2-11gr2 deinstall]$  ./deinstall -local

######################### CLEAN OPERATION END #########################

####################### CLEAN OPERATION SUMMARY #######################

Following RAC listener(s) were de-configured successfully: LISTENER

Oracle Clusterware is stopped and successfully de-configured on node "node2-11gr2"

Oracle Clusterware is stopped and de-configured successfully.

Successfully detached Oracle home '/u01/11.2.0/grid' from the central inventory on the local node.

Failed to delete directory '/u01/11.2.0/grid' on the local node.

Successfully deleted directory '/u01/app/oraInventory' on the local node.

Failed to delete directory '/u01/app/oracle' on the local node.

Oracle Universal Installer cleanup completed with errors.

Run 'rm -rf /etc/oraInst.loc' as root on node(s) 'node2-11gr2' at the end of the session.

Run 'rm -rf /opt/ORCLfmap' as root on node(s) 'node2-11gr2' at the end of the session.

Run 'rm -rf /etc/oratab' as root on node(s) 'node2-11gr2' at the end of the session.

Oracle deinstall tool successfully cleaned up temporary directories.

#######################################################################

############# ORACLE DEINSTALL & DECONFIG TOOL END #############

在保留节点执行，更新节点列表
保留节点，节点1执行
su - grid
cd $Grid_home/oui/bin
$ ./runInstaller -updateNodeList ORACLE_HOME=Grid_home "CLUSTER_NODES=
{remaining_nodes_list}" CRS=TRUE -silent

[grid@node1-11gr2 bin]$  ./runInstaller -updateNodeList ORACLE_HOME=/u01/11.2.0/grid  "CLUSTER_NODES={node1-11gr2,node3-11gr2,node4-11gr2}" CRS=TRUE -silent

 

Starting Oracle Universal Installer...

Checking swap space: must be greater than 500 MB.   Actual 7658 MB    Passed

The inventory pointer is located at /etc/oraInst.loc

The inventory is located at /u01/app/oraInventory

'UpdateNodeList' was successful.

2.4   验证是否删除成功

[grid@node1-11gr2 bin]$  cluvfy stage -post nodedel -n node2-11gr2

Performing post-checks for node removal

Checking CRS integrity...

Clusterware version consistency passed

CRS integrity check passed

Node removal check passed

Post-check for node removal was successful.

3.  异常删除RAC节点

dm01db01为要添加的节点，dm01db02 为存活节点

3.1   禁止监听

Disable the listener that runs on the failed database server:

[oracle@surviving]$ srvctl disable listener -n dm01db01

[oracle@surviving]$ srvctl stop listener -n dm01db01

PRCC-1017 : LISTENER was already stopped on dm01db01

3.2   更新Oracle用户Oracle inventory

Delete the Oracle Home from the Oracle inventory:

[oracle@surviving]$ cd ${ORACLE_HOME}/oui/bin

[oracle@surviving]$ ./runInstaller -updateNodeList

ORACLE_HOME=/u01/app/oracle/product/11.2.0/dbhome_1 "CLUSTER_NODES=dm01db02"

Starting Oracle Universal Installer...

Checking swap space: must be greater than 500 MB. Actual 16383 MB

Passed

The inventory pointer is located at /etc/oraInst.loc

The inventory is located at /u01/app/oraInventory

'UpdateNodeList' was successful.

--执行日志

[oracle@node1-11gr2 bin]$ ./runInstaller -updateNodeList ORACLE_HOME=/u01/app/oracle/product/11.2.0/db_1  "CLUSTER_NODES={node1-11gr2,node3-11gr2,node4-11gr2}"

Starting Oracle Universal Installer...

Checking swap space: must be greater than 500 MB.   Actual 7043 MB    Passed

The inventory pointer is located at /etc/oraInst.loc

The inventory is located at /u01/app/oraInventory

'UpdateNodeList' was successful.

[grid@node3-11gr2 ContentsXML]$ cat inventory.xml

<?xml version="1.0" standalone="yes" ?>

<!-- Copyright (c) 1999, 2013, Oracle and/or its affiliates.

All rights reserved. -->

<!-- Do not modify the contents of this file by hand. -->

<INVENTORY>

<VERSION_INFO>

   <SAVED_WITH>11.2.0.4.0</SAVED_WITH>

   <MINIMUM_VER>2.1.0.6.0</MINIMUM_VER>

</VERSION_INFO>

<HOME_LIST>

<HOME NAME="Ora11g_gridinfrahome1" LOC="/u01/11.2.0/grid" TYPE="O" IDX="1" CRS="true">

   <NODE_LIST>

      <NODE NAME="node1-11gr2"/>

      <NODE NAME="node3-11gr2"/>

      <NODE NAME="node4-11gr2"/>

      <NODE NAME="node2-11gr2"/>

   </NODE_LIST>

</HOME>

<HOME NAME="OraDb11g_home1" LOC="/u01/app/oracle/product/11.2.0/db_1" TYPE="O" IDX="2">

   <NODE_LIST>

      <NODE NAME="node1-11gr2"/>

      <NODE NAME="node3-11gr2"/>

      <NODE NAME="node4-11gr2"/>

   </NODE_LIST>

</HOME>

</HOME_LIST>

<COMPOSITEHOME_LIST>

备注：Oracle home 节点 已更新，变成3个节点了

3.3   验证数据库

Verify that the failed database server is unpinned:

[oracle@surviving]$ olsnodes -s -t

dm01db01 Inactive Unpinned

dm01db02 Active Unpinned

3.4   停止和移除VIP

Stop the VIP Resources for the failed database server and delete:

[root@surviving]# srvctl stop vip -i dm01db01-vip

PRCC-1016 : dm01db01-vip.acme.com was already stopped

[root@surviving]# srvctl remove vip -i dm01db01-vip

Please confirm that you intend to remove the VIPs dm01db01-vip (y/[n]) y

3.5   集群中删除节点

Delete the node from the cluster:

[root@surviving]# crsctl delete node -n dm01db01

CRS-4661: Node dm01db01 successfully deleted.

3.6   更新grid 用户Oracle Inventory

Update the Oracle Inventory:

[oracle@surviving]$ cd ${ORACLE_HOME}/oui/bin

[oracle@surviving]$

./runInstaller -updateNodeList ORACLE_HOME=/u01/app/11.2.0/grid

"CLUSTER_NODES=dm01db02" CRS=TRUE

Starting Oracle Universal

Installer...

Checking swap space: must be greater than 500 MB. Actual 16383 MB

Passed

The inventory pointer is located at /etc/oraInst.loc

The inventory is located at /u01/app/oraInventory

'UpdateNodeList' was successful.

3.7   验证节点删除成功

Verify the node deletion is successful:

[oracle@surviving]$ cluvfy stage -post nodedel -n dm01db01 -verbose

Performing post-checks for node removal

Checking CRS integrity...

The Oracle clusterware is healthy on node "dm01db02"

CRS integrity check passed

Result:

Node removal check passed

Post-check for node removal was successful.