RHEL 7.4装Oracle RAC的一些问题

cat /etc/redhat-release
Red Hat Enterprise Linux Server release 7.4 (Maipo)



yum install compat-libcap1.x86_64


clscfg.bin: error while loading shared libraries: libcap.so.1: cannot open shared object file: No such file or directory Failed to create keys in the OLR, rc = 127, Message:

Failed to write the checkpoint:” with status:FAIL.Error code is 256


二、 在执行root.sh前要给GRID_HOME打补丁 ,补丁号18370031 否则因为RHEL 7修改了一些RC脚本会导致安装不上



The following environment variables are set as:
ORACLE_HOME=  /u01/app/11.2.0/grid
Enter the full pathname of the local bin directory: [/usr/local/bin]:
The contents of “dbhome” have not changed. No need to overwrite.
The contents of “oraenv” have not changed. No need to overwrite.
The contents of “coraenv” have not changed. No need to overwrite.
Creating /etc/oratab file…
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
Using configuration parameter file: /u01/app/11.2.0/grid/crs/install/crsconfig_params
Creating trace directory
User ignored Prerequisites during installation
Installing Trace File Analyzer
OLR initialization – successful
root wallet
root wallet cert
root cert export
peer wallet
profile reader wallet
pa wallet
peer wallet keys
pa wallet keys
peer cert request
pa cert request
peer cert
pa cert
peer root cert TP
profile reader root cert TP
pa root cert TP
peer pa cert TP
pa peer cert TP
profile reader pa cert TP
profile reader peer cert TP
peer user cert
pa user cert
Adding Clusterware entries to inittab
ohasd failed to start
Failed to start the Clusterware. Last 20 lines of the alert log follow:
2018-03-08 10:20:24.544:
[client(17856)]CRS-2101:The OLR was formatted using version 3.


因为补丁18370031(Patch 18370031: RC SCRIPTS (/ETC/RC.D/RC.* , /ETC/INIT.D/* ) ON OL7 FOR CLUSTERWARE)   被 20433339( Patch 24333766: MERGE REQUEST ON TOP OF FOR BUGS 18370031 20954311)
包含了, 所以目前应该只能下载到20433339了。




opatch napply -local 20433339

opatch lsinventory
Oracle Interim Patch Installer version
Copyright (c) 2020, Oracle Corporation.  All rights reserved.

Oracle Home       : /u01/app/11.2.0/grid
Central Inventory : /u01/app/oraInventory
   from           : /u01/app/11.2.0/grid/oraInst.loc
OPatch version    :
OUI version       :
Log file location : /u01/app/11.2.0/grid/cfgtoollogs/opatch/opatch2020-03-06_09-37-02AM_1.log

Lsinventory Output file location : /u01/app/11.2.0/grid/cfgtoollogs/opatch/lsinv/lsinventory2020-03-06_09-37-02AM.txt
Local Machine Information::
Hostname: rac1
ARU platform id: 226
ARU platform description:: Linux x86-64

Installed Top-level Products (1): 

Oracle Grid Infrastructure 11g                             
There are 1 products installed in this Oracle Home.

Interim patches (1) :

Patch  24333766     : applied on Fri Mar 06 08:15:04 EST 2020
Unique Patch ID:  20433339
Patch description:  "OCW Interim patch for 24333766"
   Created on 30 Nov 2016, 12:56:34 hrs PST8PDT
   Bugs fixed:
     18370031, 20954311


OPatch succeeded.



三、安装fuser 命令

yum install psmisc



四、对于RHEL 7下的udev配置可以看这篇文章 https://gruffdba.wordpress.com/2017/02/20/udev-rules-for-asm-disks-on-rhel7/





On this blog and elsewhere you will find UDEV rules examples for setting device ownership and naming consistency on older versions of Linux.

With RHEL7 some of the syntax has changed slightly.

This example was created using OEL7 with the Red Hat kernel, but should also work on Red Hat and CentOS.

First, log in as root and check the block device is visible on the Linux host:

[root@unirac02 ~]# ls /dev/sd*
/dev/sda /dev/sda1 /dev/sda2 /dev/sdb /dev/sdb1
In this example I have created a device sdb, and as you can see I have created a partition header on it.

Next, make sure we can see the device’s SCSI ID:

[root@unirac02 ~]# /lib/udev/scsi_id -g -u /dev/sdb
Next, we are going to create a UDEV rule for this SCSI ID in the file /etc/udev/rules.d/99-oracleasm.rules.

[root@unirac01 ~]# cat /etc/udev/rules.d/99-oracleasm.rules
KERNEL=="sd?", ENV{ID_SERIAL}=="36006016004503e0017f99d58603c7c1e", SYMLINK+="oracleasm/grid1", OWNER="oracle", GROUP="oinstall", MODE="0660"
If you have several devices to add, you can use the following script to automate the rule generation.

[root@unirac02 ~]# mydevs="sdb sdc sdd" ; export count=0 ; for mydev in $mydevs; do ((count+=1)) ; /lib/udev/scsi_id -g -u /dev/$mydev | awk '{print "KERNEL==\"sd?\", ENV{ID_SERIAL}==\""$1"\", SYMLINK+=\"oracleasm/disk"ENVIRON["count"]"\", OWNER=\"oracle\", GROUP=\"oinstall\", MODE=\"0660\""}' ; done
KERNEL=="sd?", ENV{ID_SERIAL}=="36006016004503e0017f99d58603c7c1e", SYMLINK+="oracleasm/disk1", OWNER="oracle", GROUP="oinstall", MODE="0660"
KERNEL=="sd?", ENV{ID_SERIAL}=="36006016004503e0017f99d58603d1a87", SYMLINK+="oracleasm/disk2", OWNER="oracle", GROUP="oinstall", MODE="0660"
KERNEL=="sd?", ENV{ID_SERIAL}=="36006016004503e0017f99d58603d246a", SYMLINK+="oracleasm/disk3", OWNER="oracle", GROUP="oinstall", MODE="0660"
With RHEL7, restarting the UDEV rules is slightly difference than previous releases:

[root@unirac02 ~]# /sbin/udevadm control --reload-rules
[root@unirac02 ~]# /sbin/udevadm trigger
Now check, and a new device should be visible under /dev/oracleasm

[root@unirac02 ~]# ls -al /dev/oracleasm/*
lrwxrwxrwx. 1 root root 6 Feb 20 18:38 /dev/oracleasm/grid1 -> ../sdb



Requirements for Installing Oracle RDBMS on OL7 or RHEL7 64-bit (x86-64) (Doc ID 1962100.1)



Oracle Database – Standard Edition – Version to [Release 11.2]
Oracle Database – Enterprise Edition – Version to [Release 11.2]
Oracle Database Cloud Schema Service – Version N/A and later
Oracle Database Exadata Cloud Machine – Version N/A and later
Oracle Cloud Infrastructure – Database Service – Version N/A and later
Linux x86-64


This note explains the requirements that need to be met for a successful installation of Oracle 11gR2 RDBMS release on Red Hat Enterprise Linux 7.0 (or higher 7.x version), 64-bit (x86-64).  These guidelines apply to cluster (RAC) or standalone / single instances.

It is NOT the purpose of this NOTE to repeat every “how-to” step that is presented in the 11gR2 Installation Guide manual. For example this NOTE does not include how to create the Linux OS account named “oracle”, nor does it cover how to set environment variables. Both are adequately covered in Chapter 2 “Oracle Database Pre-installation Requirements” of the 11gR2 Installation Guide manual.

You can download Oracle software from My Oracle Support (patch 13390677)


This procedure is meant for those planning/installing Oracle 11gR2 RDBMS release (or higher 11.2.0.x version) on Red Hat Enterprise Linux 7.0 (or higher 7.x version) on the 64-bit (x86-64) platform. Since it is the expressed goal to keep Oracle Linux (OL) functionally IDENTICAL to RHEL, this NOTE is also completely applicable to 64-bit (x86-64) OL 7.0 (or higher 7.x version).

This procedure is not meant for those planning/installing Grid Infrastructure (GI) or any other Oracle products.


Requirements for installing Oracle 11gR2 RDBMS release 64-bit on RHEL7 or OL7 64-bit (x86_64)

I. Hardware:
1. Minimum Hardware Requirements
a.) At least 1.0 GB (1024MB) of physical RAM
b.) Swap disk space proportional to the system’s physical memory as follows:


RAM Swap Space
Between 1 GB and 2 GB 1.5 times the size of RAM
Between 2 GB and 16 GB Equal to the size of RAM
More than 16 GB 16 GB


NOTE: The above recommendations (from the 11.2 Database installation guide) are MINIMUM recommendations for installations. Further RAM and swap space may be required to tune/improve RDBMS performance.

c.) 1.0 GB (1024MB) of disk space (and less than 2TB of disk space) in the /tmp directory.
d.) approximately 4.4 GB of local disk space for the database software.
e.) approximately 1.7 GB of disk space for a preconfigured database that uses file system storage (optional).

2. Refer Note:236826.1 for details on certified filesystems for Oracle Database.

II. Software:
1. As is specified in section 1.3.2 of the Oracle Database Installation Guide for 11gR2 on Linux (part number E24321-02), Oracle recommends that you install the Linux operating system with the default software packages (RPMs) and do not customize the RPMs during installation. For additional information on “default-RPMs”, please see Note 376183.1, “Defining a “default RPMs” installation of the RHEL OS” or Note 401167.1, “Defining a “default RPMs” installation of the Oracle Enterprise Linux (OEL) OS”.

2.Linux Kernel Requirements

Oracle Linux 7.0 

  • Oracle Linux 7 with Unbreakable Enterprise Kernel : 3.8.13-33.el7uek.x86_64 or later
  • Oracle Linux 7 with the Red Hat Compatible kernel : 3.10.0-54.0.1.el7.x86_64 or later

Red Hat Enterprise Linux Server 7.0

  • Red Hat Enterprise Linux 7 : 3.10.0-54.0.1.el7.x86_64 or later


  • RHEL7 servers must be running Red Hat kernel 3.10.0-54.0.1.el7 (x86_64) or higher or 3.8.13-33.el7uek (x86_64) or higher with UEK kernel. OL7 servers must also be running kernel 3.8.13-33.el7uek (x86_64) or higher version. The product RHEL does not deliver UEK Kernel. Only in OL 7 UEK and RHCK Kernel is included.
  • It is observed there are hang issues in RHEL 7 with many CPU cores and more RAM, due NUMA was enabled. As a work around it is recommended to turn off NUMA.

3. Required OS Components (per Release Notes, and Install Guide)

a.) The exact version number details of this list are based upon 64-bit (x86_64) RHEL 7.0. When a higher “update” level is used, the RPM release numbers (such as 4.4.4-13) may be slightly different. Since updates of RHEL 7 are certified, this is fine so long as you are still using 64-bit Linux (x86_64) RHEL 7 RPMs.
b.) Some of the Install Guide requirements will already be present from the “default-RPMs” foundation of Linux that you started with:



4. Additional Required OS Components (per the runInstaller OUI)
a.) intentionally blank

5. Additional Required OS Components (per this NOTE)
a.) Please do not rush, skip, or minimize this critical step. This list is based upon a “default-RPMs” installation of 64-bit (x86_64) RHEL 6. Additional RPMs (beyond anything known to Oracle) may be needed if a “less-than-default-RPMs” installation of 64-bit (x86_64) RHEL Server 6 is performed. For more information, please refer to Note 376183.1, “Defining a “default RPMs” installation of the RHEL OS” or Note 401167.1, “Defining a “default RPMs” installation of the Oracle Enterprise Linux (OEL) OS”.
b.) Several RPMs will be required as prerequisites to those listed in section II.3.c:  


 6. Oracle Global Customer Support has noticed a recent trend with install problems that originates from installing too many RPMs. For example:
a.) installing your own JDK version (prior to execute the Oracle Software runInstaller) is not needed on Linux, and is not recommended on Linux. A pre-existing JDK often interferes with the correct JDK that the Linux Oracle Software runInstaller will place and use.
b.) installing more than the required version of the gcc / g++ RPMs often leads to accidentally using (aka enabling or activating) the incorrect one. If you have multiple RDBMS versions installed on the same Linux machine, then you will likely have to manage multiple versions of gcc /g++ . For more information, please see Note 444084.1, “Multiple gcc / g++ Versions in Linux”

7. All of the RPMs in section II. are on the Red Hat Enterprise Linux 7 64-bit (x86_64) distribution media.

III. Environment:
1. Modify your kernel settings in /etc/sysctl.conf (RedHat) as follows. If the current value for any parameter is higher than the value listed in this table, do not change the value of that parameter. Range values (such as net.ipv4.ip_local_port_range) must match exactly. 

kernel.shmall = physical RAM size / pagesize For most systems, this will be the value 2097152. See Note 301830.1 for more information.
kernel.shmmax = 1/2 of physical RAM. This would be the value 2147483648 for a system with 4GB of physical RAM. See Note:567506.1 for more information.
kernel.shmmni = 4096
kernel.sem = 250 32000 100 128
fs.file-max = 512 x processes (for example 6815744 for 13312 processes)
fs.aio-max-nr = 1048576
net.ipv4.ip_local_port_range = 9000 65500
net.core.rmem_default = 262144
net.core.rmem_max = 4194304
net.core.wmem_default = 262144
net.core.wmem_max = 1048576

2. To activate these new settings into the running kernel space, run the “sysctl -p” command as root.

3. Set Shell Limits for the oracle User. Assuming that the “oracle” Unix user will perform the installation, do the following:

a.) Add the following settings to /etc/security/limits.conf

oracle soft nproc 2047
oracle hard nproc 16384
oracle soft nofile 1024
oracle hard nofile 65536
oracle soft stack 10240

b.) Verify the latest version of PAM is loaded, then add or edit the following line in the /etc/pam.d/login file, if it does not already exist: 

session required pam_limits.so

c.) Verify the current ulimits, and raise if needed.  This can be done many ways…adding the following lines to /etc/profile is the recommended method: 

if [ $USER = “oracle” ]; then
if [ $SHELL = “/bin/ksh” ]; then
ulimit -u 16384
ulimit -n 65536
ulimit -u 16384 -n 65536


4. The gcc-4.4.4 and gcc-c++-4.4.4 RPM items above will ensure that the correct gcc / g++ versions are installed. It is also required that you ensure that these correct gcc / g++ versions are active, and in-use. Ensure that the commands “gcc –version” and “g++ –version” each return “4.8.2”.


5. If any Java packages are installed on the system, unset the Java environment variables, for example JAVA_HOME.

6. The oracle account that is used to install Oracle should not have the Oracle install related variables set by default. For example setting ORACLE_HOME, PATH, LD_LIBRARY_PATH to include Oracle binaries in .profile, .login file and /etc/profile.d should be completely avoided.
a.) Setting $ORACLE_BASE (not $ORACLE_HOME) is recommended, since it eases a few prompts in the OUI runInstaller tool.
b.) Following the successful install, it is recommended to set $ORACLE_HOME, and to set $PATH to include $ORACLE_HOME/bin at the beginning of the $PATH string.

7. By default, RHEL 7 x86_64 Linux is installed with SELinux as “enforcing”. This is fine for the 11gR2 installation process. However, to subsequently run “sqlplus”, switch SELinux to the “Permissive” mode. See NOTE 454196.1, “./sqlplus: error on libnnz11.so: cannot restore segment prot after reloc” for more details.

UPDATE: Internal testing suggests that there is no problem running “sqlplus” with SELinux in “enforcing” mode on RHEL7/OL7. The problem only affects RHEL5/OL5.

8. Log in as Oracle user and start the installation as follows: 

$ ./runInstaller -ignorePrereq

a.) It is best practice not to use any form of “su” to start the runInstaller, in order to avoid potential display-related problems.
b.) When performing the installation, make sure to use the “runInstaller” version that comes with software.
c.) When performing any subsequent 11.2.0.x patchset, make sure to use the “runInstaller” version that comes with the patchset.

Known Issue :

01) The installer needs to be launched with “-ignorePrereq” option due to unpublished bug 19947777. This issue occurs since Oracle Linux 7 was not released when Oracle database was made available and hence was not certified. However, Oracle is now certified on OL7. Refer Note 1962046.1 for details.

02) Compilation fails for target ‘relink_exe’ fails with “undefined reference to symbol ‘B_DestroyKeyObject’” error and is reported in unpublished bug 19692824. The solution is to install patch 19692824 as documented in Note 1965691.1.

1. Supported distributions of the 32-bit (x86) Linux OS can run on on AMD64/EM64T and Intel Processor Chips that adhere to the x86_64 architecture
a.) Oracle 32-bit Database Server running on AMD64/EM64T with 32-bit OS is supported, but is NOT covered by this NOTE.
b.) Oracle 32-bit Database Server running on AMD64/EM64T with 64-bit OS is not certified and is not supported.
c.) Oracle 32-bit Database Client running on AMD64/EM64T with 64-bit OS is expected to be supported, but is NOT covered by this NOTE.

2. Asynchronous I/O on ext2 and ext3 file systems is supported if your scsi/fc driver supports that functionality. 

Note : Asynchronous I/O on Ext4 file system is supported with Oracle 10g onwards on OEL5.6 and later.
Reference : Oracle Linux, Filesystem & I/O Type Supportability (Note 279069.1)

3. No extra patch is required for the DIRECTIO support for x86_64.

4. No LD_ASSUME_KERNEL value should be used with the 11gR2 product.

5. The following rpm command can be used to distinguish between a 32-bit or 64-bit package.   

# rpm -qa –queryformat “%{NAME}-%{VERSION}-%{RELEASE} (%{ARCH})\n” | grep glibc-devel
Installation walk-through – Oracle Grid/RAC on Oracle Linux 7 (Doc ID 1951613.1)

Oracle Database – Enterprise Edition – Version to [Release 11.2]
Oracle Database Cloud Schema Service – Version N/A and later
Oracle Database Exadata Cloud Machine – Version N/A and later
Oracle Cloud Infrastructure – Database Service – Version N/A and later
Oracle Database Backup Service – Version N/A and later
Linux x86-64


This document aims to provide clarity on the installation/patching processes required while installing Oracle Grid and Oracle RAC on Oracle Linux 7 by providing details on the steps taken to complete an example installation. For general recommendations, refer to Note 1962100.1 “Requirements for Installing Oracle RDBMS on OL7 or RHEL7 64-bit (x86-64)”

This document is intended to complement the official Oracle documentation. If there are any incompatibilities between this document and the official Oracle documentation, they are unintentional, and should be ignored. This document is not meant to be a substitute for official documentation; care should be taken to ensure that all official documentation is reviewed thoroughly.

Operating System Installation & Setup – Recommendations

Yum Repository
Set up public-yum repository and enable the latest AddOns channels, e.g.

# cat /etc/yum.repos.d/public-yum-ol7.repo
name=Oracle Linux $releasever Latest ($basearch)

name=Oracle Linux $releasever Add ons ($basearch)


11gR2 preinstall RPM
Install 11gR2 preinstall rpm. The preinstall rpm installs all dependencies for the Oracle RDBMS server installation, and creates the oracle user and the dba and oinstall groups.

Oracle ASMLib
Download the certified oracleasmlib package from OTN (http://www.oracle.com/technetwork/server-storage/linux/asmlib/ol7-2352094.html)

Install oracleasm packages if oracleasmlib is to be used


yum install oracleasm-support.x86_64 oracleasmlib-2.0.8-2.el7.x86_64.rpm


Oracle Automatic Storage Management Cluster File System (Oracle ACFS)
For details on ACFS support, including required patches, refer to Note 1369107.1 (ACFS Support On OS Platforms (Certification Matrix)

Disk Naming Consistency
For consistent disk naming, install device-mapper-multipath along with associated dependencies (device-mapper-multipath-libs.x86_64)


yum install device-mapper-multipath



Start multipathd, and verify status e.g.


[root@xxxx ~]# systemctl start multipathd
[root@xxxx ~]# systemctl status multipathd

Disable, e.g.

setenforce 0

Oracle Grid Infrastructure – Installation Notes
Patch 19404309

Note: It is presumed that the user has already reviewed the Oracle Grid Infrastructure Installation Guide and associated Release Notes; instructions and/or recommendations from those documents will not be repeated here.

After downloading the Oracle Grid Infrastructure software, and before attempting any installation, download Patch 19404309 from My Oracle Support, and apply the patch using the instructions in the patch README.

Patch 18370031

Download Patch 18370031 from My Oracle Support. Then, start an interactive Oracle Grid Infrastructure installation through the Oracle Universal Installer (OUI), but do not execute root.sh on any node until afterthe application of Patch 18370031. When the OUI prompts the user to execute the root.sh scripts*, Patch 18370031 should be applied by following the instructions in Section 2.3, Case 5 – Patching a Software Only GI Home Installation or Before the GI Home Is Configured – of the patch README. Note: The README should be reviewed in full, as it contains other requirements (e.g. upgrading OPatch, etc.).

* If executing a software-only installation, the patch should be applied after the installation concludes, but before any configuration is attempted.

Once Patch 18370031 has been applied, proceed with the remainder of the installation (or configuration).

Oracle Database/RAC – Installation Notes
Note: As the title suggests, this section applies both to installations of Oracle Database and Oracle Real Application Clusters (RAC).

Patch 19404309
Note: It is presumed that the user has already reviewed the Oracle Database, Oracle RAC Installation Guides and associated Release Notes; instructions and/or recommendations from those documents will not be repeated here.

After downloading the Oracle Database/RAC software, and before attempting any installation, download Patch 19404309 from My Oracle Support, and apply the patch using the instructions in the patch README.

Patch 19692824
During installation of Oracle Database or Oracle RAC on OL7, the following linking error may be encountered:


Error in invoking target ‘agent nmhs’ of makefile ‘<ORACLE_HOME>/sysman/lib/ins_emagent.mk’. See ‘<installation log>’ for details.
If this error is encountered, the user should select Continue. Then, after the installation has completed, the user must download Patch 19692824 from My Oracle Support and apply it per the instructions included in the patch README.

Installation/Home Cloning
Note: It may be possible to perform the above steps once, then use Oracle’s cloning technology to clone the installation/home. Further details are available in the cloning sections of the relevant Administration and Deployment guides:

Cloning Oracle Clusterware

Cloning Oracle RAC to Nodes in a New Cluster

Cloning Oracle Software






2 responses to “RHEL 7.4装Oracle RAC的一些问题”

  1. 不了峰 Avatar

    第一个节点执行 执行root.sh 报错的另一个解决方法是:
    当提示出错时,[client(17856)]CRS-2101:The OLR was formatted using version 3.
    在RHEL 7中ohasd需要被设置为一个服务,在运行脚本root.sh之前。

    touch /usr/lib/systemd/system/ohas.service
    chmod 777 /usr/lib/systemd/system/ohas.service

    vi /usr/lib/systemd/system/ohas.service

    Description=Oracle High Availability Services

    ExecStart=/etc/init.d/init.ohasd run >/dev/null 2>&1 Type=simple


    systemctl daemon-reload
    systemctl enable ohas.service
    systemctl start ohas.service

    systemctl status ohas.service
    要保障服务启动后, root.sh会自动继续下去

  2. 不了峰 Avatar


Leave a Reply

Your email address will not be published. Required fields are marked *