Oracle Patching Tips

*********************************************************************************************
-- 1. Download and Unzip the Latest OPatch (Specific to Database Release) to all cluster nodes:
*********************************************************************************************
You must use the OPatch utility version 11.2.0.3.6 or later to apply this patch. Oracle recommends that you use the latest released OPatch for 11.2 releases, which is available for download from My Oracle Support patch 6880880.
It is always a best practice to keep both GRID_HOME and DATABASE HOME at same opatch level. So change the opatch in DATABASE HOME also.
Repeat this step to all the nodes in the cluster.

OPatch patch of version 11.2.0.3.6 for Oracle software releases 11.2.0.x (DEC 2013) (Patch)
p6880880_112000_AIX64-5L.zip (Unzip the file in a shared location

i.e. /sw61/ucms/Oracle11R2Stage/patch)

--***************************************************************
-- 2. Validate and Record Pre-Patch information :
--***************************************************************
Validate using the following commands :
$ cd $GRID_HOME/OPatch
$ opatch lsinventory -detail -oh /u01/products/grid_11203/
$ cd $ORACLE_HOME/OPatch
$ opatch lsinventory -detail –oh /u01/products/rdbms_11203
Login to each node in RAC as grid user and execute the following command.
$ cd $GRID_HOME/OPatch
$ opatch lsinventory
$ opatch lsinventory -bugs_fixed | grep -i 'PSU'
Login to each node in RAC as oracle user and execute the following command.
$ cd $ORACLE_HOME/OPatch
$ opatch lsinventory
$ opatch lsinventory -bugs_fixed | grep -i 'PSU'
Connect to each instance and record registry information.
$ sqlplus / as sysdba
set linesize 500
set pagesize 500
column  comp_name format a40
column  version format a15
column  status format a13
select comp_name,version,status from dba_registry
/
SQL> exit;

--***************************************************************
-- 3. Create OCM Response File If It Does Not Exist :
--***************************************************************
Create ocm response file using the following command and provide appropriate values for the prompts.
$ cd $GRID_HOME/OPatch/ocm/bin/
$ emocmrsp
Verify the created file using,
$ emocmrsp –verbose ocm.rsp
NOTE: The Opatch utility will prompt for your OCM (Oracle Configuration Manager) response file when it is run. Without which we cant proceed further.
Copy this response file ocm.rsp to all the nodes in the cluster to the same location. Or you can create a new response file on each node using the same method above.

--***************************************************************
-- 4. Download and Unzip the JUL2014 PSU patch : (as grid user)
--***************************************************************
Use the following Metalink Note to locate the correct patch to download:
Oracle Recommended Patches -- Oracle Database (Doc ID 756671.1)
GRID INFRASTRUCTURE PATCH SET UPDATE 11.2.0.3.11 (INCLUDES DB PSU 11.2.0.3.11) (Patch)
p18706488_112030_AIX64-5L.zip (Unzip the file in a shared location i.e. /sw61/ucms/Oracle11R2Stage/patch)

**********************************************************************************
-- 5. Stop EM Agent Processes Prior to Patching and Prior to Rolling Back the Patch
**********************************************************************************

You must stop the EM agent processes running from the database home, prior to patching the Oracle RAC database or GI Home and prior to rolling back the patch from Oracle RAC database or GI Home. Execute the following command on the node to be patched or the node where the patch is to be rolled back.

As the Oracle RAC database home owner execute:
 $ <ORACLE_HOME>/bin/emctl stop dbconsole

--***************************************************************
-- 6. One-off patch Conflicts detection and Resolution :
--***************************************************************
Determine whether any currently installed one-off patches conflict with the PSU patch.
$ cd $GRID_HOME/OPatch
$ opatchauto apply <UNZIPPED_PATCH_LOCATION>/18706488  analyze
If you have any conflicts identified in GI home or in DB home, follow (Patch Set Updates - One-off Patch Conflict Resolution (Doc ID 1061295.1)) to resolve the conflicts.

--***************************************************************
-- 7. Patch Application :
--***************************************************************
Now patch can be applied using the following syntax.
As root user, execute the following command on each node of the cluster:
# opatch auto <UNZIPPED_PATCH_LOCATION> -ocmrf <ocm response file>
Refer to the following Metalink Note to apply as a rolling patch. 18706488 can be applied as a rolling patch.
Rolling Patch - OPatch Support for RAC (Doc ID 244241.1)
You can choose from the following three methods.
All-Node Patch:
  Shutdown all Oracle instances on all nodes
  Apply the patch to the RAC home on all nodes
  Bring all instances up
Minimum downtime:
 Shutdown the Oracle instance on node 1
 Apply the patch to the RAC home on node 1
 Shutdown the Oracle instance on node 2
 Apply the patch to the RAC home on node 2
 Shutdown the Oracle instance on node 3
 At this point, instances on nodes 1 and 2 can be brought up
 Apply the patch to the RAC home on node 3
 Startup the Oracle instance on node 3
Rolling patch (no downtime):
 Shutdown the Oracle instance on node 1
 Apply the patch to the RAC home on node 1
 Start the Oracle instance on node 1
 Shutdown the Oracle instance on node 2
 Apply the patch to the RAC home on node 2
 Start the Oracle instance on node 2
 Shutdown the Oracle instance on node 3
 Apply the patch to the RAC home on node 3
 Start the Oracle instance on node 3

--***************************************************************
-- 8. Upgrade Oracle Recovery Manager Catalog
--***************************************************************

If you are using the Oracle Recovery Manager, the catalog needs to be upgraded. Enter the following command to upgrade it:
$ rman target / catalog rman/<password>@UCRMAN
RMAN> UPGRADE CATALOG;

--***************************************************************
-- 9. Verification of Patch application :
--***************************************************************

Validate using the following commands :
$ cd $GRID_HOME/OPatch
$ opatch lsinventory -detail -oh /u01/products/grid_11203/
$ cd $ORACLE_HOME/OPatch
$ opatch lsinventory -detail –oh /u01/products/rdbms_11203
Login to each node in RAC as grid user and execute the following command.
$ cd $GRID_HOME/OPatch
$ opatch lsinventory
$ opatch lsinventory -bugs_fixed | grep -i 'PSU'
Login to each node in RAC as oracle user and execute the following command.
$ cd $ORACLE_HOME/OPatch
$ opatch lsinventory
$ opatch lsinventory -bugs_fixed | grep -i 'PSU'
Connect to each instance and record registry information.
$ sqlplus / as sysdba
set linesize 500
set pagesize 500
column  comp_name format a40
column  version format a15
column  status format a13
select comp_name,version,status from dba_registry
/
SQL> exit;

--***************************************************************************
-- 10. (If Required) Roll Back the Oracle RAC Database Homes and GI Together
--***************************************************************************
GI Home and Database Homes that are not shared and ACFS file system is not configured.

As root user, execute the following command on each node of the cluster.
# cd $GRID_HOME/OPatch
# opatch auto <UNZIPPED_PATCH_LOCATION> -rollback -ocmrf <ocm response file>