Ask Maclean

在11gR2 RAC中修改ASM DISK Path磁盘路径

有同学在T.askmac.cn上提问关于修改11gR2中ASM DISK的路径问题，具体问题如下：

 

aix 6.1，grid 11.2.0.3+asm11.2.0.3+rac

建数据库的时候使用的是aix自带的多路径软件mpio，建了diskgroup

现在改造成veritas dmp多路径，已经修改了asm的disk_strings＝/dev/vx/rdmp/*，crs/asm启动的时候已经可以识别到磁盘/dev/vx/rdmp/开头的磁盘，但是读取不回原来的diskgroup信息。

crs启动的报错日志：
2012-07-13 15:07:29.748: [ GPNP][1286]clsgpnp_profileCallUrlInt: [at clsgpnp.c:2108 clsgpnp_profileCallUrlInt] get-profile call to url “ipc://GPNPD_ggtest1” disco “” [f=0 claimed- host: cname: seq: auth:]
2012-07-13 15:07:29.762: [ GPNP][1286]clsgpnp_profileCallUrlInt: [at clsgpnp.c:2236 clsgpnp_profileCallUrlInt] Result: (0) CLSGPNP_OK. Successful get-profile CALL to remote “ipc://GPNPD_ggtest1” disco “”
2012-07-13 15:07:29.762: [ CSSD][1286]clssnmReadDiscoveryProfile: voting file discovery string(/dev/vx/rdmp/*)
2012-07-13 15:07:29.762: [ CSSD][1286]clssnmvDDiscThread: using discovery string /dev/vx/rdmp/* for initial discovery
2012-07-13 15:07:29.762: [ SKGFD][1286]Discovery with str:/dev/vx/rdmp/*:

2012-07-13 15:07:29.762: [ SKGFD][1286]UFS discovery with :/dev/vx/rdmp/*:

2012-07-13 15:07:29.769: [ SKGFD][1286]Fetching UFS disk :/dev/vx/rdmp/v_df8000_919:

2012-07-13 15:07:29.770: [ SKGFD][1286]Fetching UFS disk :/dev/vx/rdmp/v_df8000_212:

2012-07-13 15:07:29.770: [ SKGFD][1286]Fetching UFS disk :/dev/vx/rdmp/v_df8000_211:

2012-07-13 15:07:29.770: [ SKGFD][1286]Fetching UFS disk :/dev/vx/rdmp/v_df8000_210:

2012-07-13 15:07:29.770: [ SKGFD][1286]Fetching UFS disk :/dev/vx/rdmp/v_df8000_209:

2012-07-13 15:07:29.771: [ SKGFD][1286]Fetching UFS disk :/dev/vx/rdmp/v_df8000_181:

2012-07-13 15:07:29.771: [ SKGFD][1286]Fetching UFS disk :/dev/vx/rdmp/v_df8000_180:

2012-07-13 15:07:29.771: [ SKGFD][1286]Fetching UFS disk :/dev/vx/rdmp/disk_3:

2012-07-13 15:07:29.771: [ SKGFD][1286]Fetching UFS disk :/dev/vx/rdmp/disk_2:

2012-07-13 15:07:29.771: [ SKGFD][1286]Fetching UFS disk :/dev/vx/rdmp/disk_1:

2012-07-13 15:07:29.771: [ SKGFD][1286]Fetching UFS disk :/dev/vx/rdmp/disk_0:

2012-07-13 15:07:29.771: [ SKGFD][1286]OSS discovery with :/dev/vx/rdmp/*:

2012-07-13 15:07:29.771: [ SKGFD][1286]Handle 1115e7510 from lib :UFS:: for disk :/dev/vx/rdmp/v_df8000_916:

2012-07-13 15:07:29.772: [ SKGFD][1286]Handle 1118758b0 from lib :UFS:: for disk :/dev/vx/rdmp/v_df8000_912:

2012-07-13 15:07:29.773: [ SKGFD][1286]Handle 1118d9cf0 from lib :UFS:: for disk :/dev/vx/rdmp/v_df8000_908:

2012-07-13 15:07:29.773: [ SKGFD][1286]Handle 1118da450 from lib :UFS:: for disk :/dev/vx/rdmp/v_df8000_904:

2012-07-13 15:07:29.773: [ SKGFD][1286]Handle 1118dad70 from lib :UFS:: for disk :/dev/vx/rdmp/v_df8000_903:

2012-07-13 15:07:29.802: [ CLSF][1286]checksum failed for disk:/dev/vx/rdmp/v_df8000_916:
2012-07-13 15:07:29.803: [ SKGFD][1286]Lib :UFS:: closing handle 1115e7510 for disk :/dev/vx/rdmp/v_df8000_916:

2012-07-13 15:07:29.803: [ SKGFD][1286]Lib :UFS:: closing handle 1118758b0 for disk :/dev/vx/rdmp/v_df8000_912:

2012-07-13 15:07:29.804: [ SKGFD][1286]Handle 1115e6710 from lib :UFS:: for disk :/dev/vx/rdmp/v_df8000_202:

2012-07-13 15:07:29.808: [ SKGFD][1286]Handle 1115e7030 from lib :UFS:: for disk :/dev/vx/rdmp/v_df8000_201:

2012-07-13 15:07:29.809: [ SKGFD][1286]Handle 1115e7ad0 from lib :UFS:: for disk :/dev/vx/rdmp/v_df8000_200:

2012-07-13 15:07:29.809: [ SKGFD][1286]Handle 1118733f0 from lib :UFS:: for disk :/dev/vx/rdmp/v_df8000_199:

2012-07-13 15:07:29.816: [ CLSF][1286]checksum failed for disk:/dev/vx/rdmp/v_df8000_186:
2012-07-13 15:07:29.816: [ SKGFD][1286]Lib :UFS:: closing handle 1118de5d0 for disk :/dev/vx/rdmp/v_df8000_186:

2012-07-13 15:07:29.816: [ CSSD][1286]clssnmvDiskVerify: Successful discovery of 0 disks
2012-07-13 15:07:29.816: [ CSSD][1286]clssnmCompleteInitVFDiscovery: Completing initial voting file discovery
2012-07-13 15:07:29.816: [ CSSD][1286]clssnmvFindInitialConfigs: No voting files found
2012-07-13 15:07:29.816: [ CSSD][1286](:CSSNM00070:)clssnmCompleteInitVFDiscovery: Voting file not found. Retrying discovery in 15 seconds
2012-07-13 15:07:30.169: [ CSSD][1029]clssgmExecuteClientRequest(): type(37) size(80) only connect and exit messages are allowed before lease acquisition proc(1115e4870) client(0)

 

 

其需求为修改ASM DISK PATH的磁盘设备路径，但是由于11gR2 RAC+ASM的特殊性，导致CRS无法正常启动，虽然使用crsctl start crs -excl -nocrs的方式可以启动CSS服务和ASM实例，

 

但是最后还是报(clssnmCompleteInitVFDiscovery: Voting file not found)，这是由于Voteing file投票磁盘的位置没有合理更新导致的。

 

 

这里我们来学习一下，如何正确的修改11gR2 RAC+ASM下的ASM DISK路径：

 

 

1.我们来营造一个修改ASM DISK路径的环境，这里我们使用UDEV的设备名绑定服务，并利用UDEV将原本的ASM DISK从/dev/asm-disk* 修改为 /dev/rasm-disk*的形式， 这只需要修改udev rule文件即可实现：

 

 

[grid@maclean1 ~]$ export  ORACLE_HOME=/g01/grid/app/11.2.0/grid

[grid@maclean1 ~]$ /g01/grid/app/11.2.0/grid/bin/sqlplus  / as sysasm

SQL*Plus: Release 11.2.0.3.0 Production on Sun Jul 15 04:09:28 2012

Copyright (c) 1982, 2011, Oracle.  All rights reserved.

Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production
With the Real Application Clusters and Automatic Storage Management options

SQL> show parameter diskstri 

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
asm_diskstring                       string      /dev/asm*

 

可以看到当前ASM实例使用的asm_diskstring 为/dev/asm*， 切换到root用户修改UDEV RULE文件 ：

 

 

[root@maclean1 rules.d]# cp 99-oracle-asmdevices.rules  99-oracle-asmdevices.rules.bak
[root@maclean1 rules.d]# vi 99-oracle-asmdevices.rules

[root@maclean1 rules.d]# cat 99-oracle-asmdevices.rules

KERNEL=="sd*", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -s %p", RESULT=="SATA_VBOX_HARDDISK_VB09cadb31-cfbea255_", NAME="rasm-diskb", OWNER="grid", GROUP="asmadmin", MODE="0660"
KERNEL=="sd*", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -s %p", RESULT=="SATA_VBOX_HARDDISK_VB5f097069-59efb82f_", NAME="rasm-diskc", OWNER="grid", GROUP="asmadmin", MODE="0660"
KERNEL=="sd*", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -s %p", RESULT=="SATA_VBOX_HARDDISK_VB4e1a81c0-20478bc4_", NAME="rasm-diskd", OWNER="grid", GROUP="asmadmin", MODE="0660"
KERNEL=="sd*", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -s %p", RESULT=="SATA_VBOX_HARDDISK_VBdcce9285-b13c5a27_", NAME="rasm-diske", OWNER="grid", GROUP="asmadmin", MODE="0660"
KERNEL=="sd*", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -s %p", RESULT=="SATA_VBOX_HARDDISK_VB82effe1a-dbca7dff_", NAME="rasm-diskf", OWNER="grid", GROUP="asmadmin", MODE="0660"
KERNEL=="sd*", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -s %p", RESULT=="SATA_VBOX_HARDDISK_VB950d279f-c581cb51_", NAME="rasm-diskg", OWNER="grid", GROUP="asmadmin", MODE="0660"
KERNEL=="sd*", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -s %p", RESULT=="SATA_VBOX_HARDDISK_VB14400d81-651672d7_", NAME="rasm-diskh", OWNER="grid", GROUP="asmadmin", MODE="0660"
KERNEL=="sd*", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -s %p", RESULT=="SATA_VBOX_HARDDISK_VB31b1237b-78aa22bb_", NAME="rasm-diski", OWNER="grid", GROUP="asmadmin", MODE="0660"

 

 

以上修改了原始99-oracle-asmdevices.rules的UDEV RULE规则文件，生成的设备名被修改为/dev/rasm-disk*的形式，不同于之前的ASM DISK设备名， 这要求我们后续的一系列操作来保证RAC CRS可以正常启动。

 

当前运行时的votedisk和ocr 存放位置:

 

[root@maclean1 rules.d]# /g01/grid/app/11.2.0/grid/bin/crsctl query css votedisk
##  STATE    File Universal Id                File Name Disk group
--  -----    -----------------                --------- ---------
 1. ONLINE   6896bfc3d1464f9fbf0ea9df87e023ad (/dev/asm-diskb) [SYSTEMDG]
 2. ONLINE   58eb81b656084ff2bfd315d9badd08b7 (/dev/asm-diskc) [SYSTEMDG]
 3. ONLINE   6bf7324625c54f3abf2c942b1e7f70d9 (/dev/asm-diskd) [SYSTEMDG]
 4. ONLINE   43ad8ae20c354f5ebf7083bc30bf94cc (/dev/asm-diske) [SYSTEMDG]
 5. ONLINE   4c225359d51b4f93bfba01080664b3d7 (/dev/asm-diskf) [SYSTEMDG]
Located 5 voting disk(s).

[root@maclean1 rules.d]# /g01/grid/app/11.2.0/grid/bin/ocrcheck
Status of Oracle Cluster Registry is as follows :
         Version                  :          3
         Total space (kbytes)     :     262120
         Used space (kbytes)      :       2844
         Available space (kbytes) :     259276
         ID                       :  879001605
         Device/File Name         :  +SYSTEMDG
                                    Device/File integrity check succeeded

                                    Device/File not configured

                                    Device/File not configured

                                    Device/File not configured

                                    Device/File not configured

         Cluster registry integrity check succeeded

         Logical corruption check succeeded

 

 

因为votedisk file的位置具体指向一个ASM DISK，所以后续我们会需要crsctl replace votedisk， 我们现在重启LINUX OS:

 

 

[root@maclean1 rules.d]# init 6

rebooting ............

[root@maclean1 dev]# ls -l *asm*
brw-rw---- 1 grid asmadmin 8,  16 Jul 15 04:15 rasm-diskb
brw-rw---- 1 grid asmadmin 8,  32 Jul 15 04:15 rasm-diskc
brw-rw---- 1 grid asmadmin 8,  48 Jul 15 04:15 rasm-diskd
brw-rw---- 1 grid asmadmin 8,  64 Jul 15 04:15 rasm-diske
brw-rw---- 1 grid asmadmin 8,  80 Jul 15 04:15 rasm-diskf
brw-rw---- 1 grid asmadmin 8,  96 Jul 15 04:15 rasm-diskg
brw-rw---- 1 grid asmadmin 8, 112 Jul 15 04:15 rasm-diskh
brw-rw---- 1 grid asmadmin 8, 128 Jul 15 04:15 rasm-diski

 

 

重启后自动获得了形如/dev/rasm-disk*的ASM DISK,查阅ASM日志可以发现css服务仍搜索/dev/asm*路径来获得ASM DISK，但是这将导致找不到任何有效的ASM DISK:

 

more /g01/grid/app/11.2.0/grid/log/maclean1/cssd/ocssd.log

2012-07-15 04:17:45.208: [ SKGFD][1099548992]Discovery with str:/dev/asm*:

2012-07-15 04:17:45.208: [   SKGFD][1099548992]UFS discovery with :/dev/asm*:

2012-07-15 04:17:45.208: [   SKGFD][1099548992]OSS discovery with :/dev/asm*:

2012-07-15 04:17:45.208: [ CSSD][1099548992]clssnmvDiskVerify: Successful discovery of 0 disks
2012-07-15 04:17:45.208: [    CSSD][1099548992]clssnmCompleteInitVFDiscovery: Completing initial voting file discovery
2012-07-15 04:17:45.208: [    CSSD][1099548992]clssnmvFindInitialConfigs: No voting files found
2012-07-15 04:17:45.208: [    CSSD][1099548992](:CSSNM00070:)clssnmCompleteInitVFDiscovery: Voting file not found. 
Retrying discovery in 15 seconds
2012-07-15 04:17:45.251: [    CSSD][1096661312]clssgmExecuteClientRequest(): type(37) size(80) only connect and 
exit messages are allowed before lease acquisition proc(0x26a8ba0) client((nil))
2012-07-15 04:17:45.251: [    CSSD][1096661312]clssgmDeadProc: proc 0x26a8ba0
2012-07-15 04:17:45.251: [    CSSD][1096661312]clssgmDestroyProc: cleaning up proc(0x26a8ba0) con(0xfe6) skgpid  
ospid 3751 with 0 clients, refcount 0
2012-07-15 04:17:45.252: [    CSSD][1096661312]clssgmDiscEndpcl: gipcDestroy 0xfe6
2012-07-15 04:17:45.829: [    CSSD][1096661312]clssscSelect: cookie accept request 0x2318ea0
2012-07-15 04:17:45.829: [    CSSD][1096661312]clssgmAllocProc: (0x2659480) allocated
2012-07-15 04:17:45.830: [    CSSD][1096661312]clssgmClientConnectMsg: properties of cmProc 0x2659480 - 1,2,3,4,5
2012-07-15 04:17:45.830: [    CSSD][1096661312]clssgmClientConnectMsg: Connect from con(0x114e) proc(0x2659480) pid(3751) 
version 11:2:1:4, properties: 1,2,3,4,5
2012-07-15 04:17:45.830: [    CSSD][1096661312]clssgmClientConnectMsg: msg flags 0x0000
2012-07-15 04:17:45.939: [    CSSD][1096661312]clssscSelect: cookie accept request 0x253ddd0
2012-07-15 04:17:45.939: [    CSSD][1096661312]clssscevtypSHRCON: getting client with cmproc 0x253ddd0
2012-07-15 04:17:45.939: [    CSSD][1096661312]clssgmRegisterClient: proc(3/0x253ddd0), client(61/0x26877b0)
2012-07-15 04:17:45.939: [    CSSD][1096661312]clssgmExecuteClientRequest(): type(6) size(684) only connect and exit messages are 
allowed before lease acquisition proc(0x253ddd0) client(0x26877b0)
2012-07-15 04:17:45.939: [    CSSD][1096661312]clssgmDiscEndpcl: gipcDestroy 0x1174
2012-07-15 04:17:46.070: [    CSSD][1096661312]clssscSelect: cookie accept request 0x26368a0
2012-07-15 04:17:46.070: [    CSSD][1096661312]clssscevtypSHRCON: getting client with cmproc 0x26368a0
2012-07-15 04:17:46.070: [    CSSD][1096661312]clssgmRegisterClient: proc(5/0x26368a0), client(50/0x26877b0)

 

 

由于11gR2中CRS服务依赖于ASM，因为ocr存放在ASM中，所以ASM若无法有效启动，这导致CRS服务也无法正常工作:

[root@maclean1 ~]# crsctl check has
CRS-4638: Oracle High Availability Services is online

[root@maclean1 ~]# crsctl check crs
CRS-4638: Oracle High Availability Services is online
CRS-4535: Cannot communicate with Cluster Ready Services
CRS-4530: Communications failure contacting Cluster Synchronization Services daemon
CRS-4534: Cannot communicate with Event Manager

 

 

2. 以上是修改ASM DISK PATH磁盘路径的现状，我们需要通过以下操作来恢复CRS:

首先彻底关闭OHASD服务：

 

[root@maclean1 ~]# crsctl stop has -f 

CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'maclean1'
CRS-2673: Attempting to stop 'ora.mdnsd' on 'maclean1'
CRS-2673: Attempting to stop 'ora.crf' on 'maclean1'
CRS-2677: Stop of 'ora.mdnsd' on 'maclean1' succeeded
CRS-2677: Stop of 'ora.crf' on 'maclean1' succeeded
CRS-2673: Attempting to stop 'ora.gipcd' on 'maclean1'
CRS-2677: Stop of 'ora.gipcd' on 'maclean1' succeeded
CRS-2673: Attempting to stop 'ora.gpnpd' on 'maclean1'
CRS-2677: Stop of 'ora.gpnpd' on 'maclean1' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'maclean1' has completed
CRS-4133: Oracle High Availability Services has been stopped.

 

 

3. 以-excl -nocrs方式启动CRS，这将仅启动ASM 实例而不会启动CRS服务：

 

 

 [root@maclean1 ~]# crsctl start crs -excl -nocrs 

CRS-4123: Oracle High Availability Services has been started.
CRS-2672: Attempting to start 'ora.mdnsd' on 'maclean1'
CRS-2676: Start of 'ora.mdnsd' on 'maclean1' succeeded
CRS-2672: Attempting to start 'ora.gpnpd' on 'maclean1'
CRS-2676: Start of 'ora.gpnpd' on 'maclean1' succeeded
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'maclean1'
CRS-2672: Attempting to start 'ora.gipcd' on 'maclean1'
CRS-2676: Start of 'ora.cssdmonitor' on 'maclean1' succeeded
CRS-2676: Start of 'ora.gipcd' on 'maclean1' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'maclean1'
CRS-2672: Attempting to start 'ora.diskmon' on 'maclean1'
CRS-2676: Start of 'ora.diskmon' on 'maclean1' succeeded
CRS-2676: Start of 'ora.cssd' on 'maclean1' succeeded
CRS-2679: Attempting to clean 'ora.cluster_interconnect.haip' on 'maclean1'
CRS-2672: Attempting to start 'ora.ctssd' on 'maclean1'
CRS-2681: Clean of 'ora.cluster_interconnect.haip' on 'maclean1' succeeded
CRS-2672: Attempting to start 'ora.cluster_interconnect.haip' on 'maclean1'
CRS-2676: Start of 'ora.ctssd' on 'maclean1' succeeded
CRS-2676: Start of 'ora.cluster_interconnect.haip' on 'maclean1' succeeded
CRS-2672: Attempting to start 'ora.asm' on 'maclean1'
CRS-2676: Start of 'ora.asm' on 'maclean1' succeeded	

#建议同时修改CRS_HOME所在的ORACLE_BASE为777权限，避免可能的问题
[root@maclean1 ~]# chmod 777 /g01

 

 

 

4.修改ASM实例的disk_strings为当前的ASM DISK PATH信息:

 

[root@maclean1 ~]# su - grid

[grid@maclean1 ~]$ sqlplus  / as sysasm

SQL*Plus: Release 11.2.0.3.0 Production on Sun Jul 15 04:40:40 2012

Copyright (c) 1982, 2011, Oracle.  All rights reserved.

Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production
With the Real Application Clusters and Automatic Storage Management options

SQL> alter system set asm_diskstring='/dev/rasm*';

System altered.

SQL> alter diskgroup systemdg mount;

Diskgroup altered.

SQL> create spfile from memory;

File created.

SQL> startup force mount;
ORA-32004: obsolete or deprecated parameter(s) specified for ASM instance
ASM instance started

Total System Global Area  283930624 bytes
Fixed Size                  2227664 bytes
Variable Size             256537136 bytes
ASM Cache                  25165824 bytes
ASM diskgroups mounted

SQL> show parameter spfile

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
spfile                               string      /g01/grid/app/11.2.0/grid/dbs/
                                                 spfile+ASM1.ora

SQL> show parameter disk

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
asm_diskgroups                       string      SYSTEMDG
asm_diskstring                       string      /dev/rasm*

SQL> create pfile from spfile;

File created.

SQL> create spfile='+SYSTEMDG' from pfile;

File created.

SQL> startup force;
ORA-32004: obsolete or deprecated parameter(s) specified for ASM instance
ASM instance started

Total System Global Area  283930624 bytes
Fixed Size                  2227664 bytes
Variable Size             256537136 bytes
ASM Cache                  25165824 bytes
ASM diskgroups mounted
SQL> show parameter spfile

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
spfile                               string      +SYSTEMDG/maclean-cluster/asmp
                                                 arameterfile/registry.253.7886
                                                 82933

 

 

以上成功修改了asm_diskstring ，且更新了ASM DISKGROUP上的SPFILE ， 由于ASM使用共享的SPFILE所以其他节点上一般无需在做其他操作。

 

5. crsctl replace votedisk 命令将votedisk重置位置：

 

 

[root@maclean1 ~]# crsctl replace votedisk +systemdg

Successful addition of voting disk 864a00efcfbe4f42bfd0f4f6b60472a0.
Successful addition of voting disk ab14d6e727614f29bf53b9870052a5c8.
Successful addition of voting disk 754c03c168854f46bf2daee7287bf260.
Successful addition of voting disk 9ed58f37f3e84f28bfcd9b101f2af9f3.
Successful addition of voting disk 4ce7b7c682364f12bf4df5ce1fb7814e.
Successfully replaced voting disk group with +systemdg.
CRS-4266: Voting file(s) successfully replaced		

[root@maclean1 ~]# crsctl query css votedisk
##  STATE    File Universal Id                File Name Disk group
--  -----    -----------------                --------- ---------
 1. ONLINE   864a00efcfbe4f42bfd0f4f6b60472a0 (/dev/rasm-diskb) [SYSTEMDG]
 2. ONLINE   ab14d6e727614f29bf53b9870052a5c8 (/dev/rasm-diskc) [SYSTEMDG]
 3. ONLINE   754c03c168854f46bf2daee7287bf260 (/dev/rasm-diskd) [SYSTEMDG]
 4. ONLINE   9ed58f37f3e84f28bfcd9b101f2af9f3 (/dev/rasm-diske) [SYSTEMDG]
 5. ONLINE   4ce7b7c682364f12bf4df5ce1fb7814e (/dev/rasm-diskf) [SYSTEMDG]
Located 5 voting disk(s).

[root@maclean1 ~]# ocrcheck
Status of Oracle Cluster Registry is as follows :
         Version                  :          3
         Total space (kbytes)     :     262120
         Used space (kbytes)      :       2844
         Available space (kbytes) :     259276
         ID                       :  879001605
         Device/File Name         :  +SYSTEMDG
                                    Device/File integrity check succeeded

                                    Device/File not configured

                                    Device/File not configured

                                    Device/File not configured

                                    Device/File not configured

         Cluster registry integrity check succeeded

         Logical corruption check succeeded

 

 

以上replace了votedisk到新的 ASM DISK上，并确认votedisk和OCR均为可用状态。

 

6.重启CRS服务：

 

 

[root@maclean1 ~]# crsctl stop crs
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'maclean1'
CRS-2673: Attempting to stop 'ora.mdnsd' on 'maclean1'
CRS-2673: Attempting to stop 'ora.ctssd' on 'maclean1'
CRS-2673: Attempting to stop 'ora.asm' on 'maclean1'
CRS-2677: Stop of 'ora.mdnsd' on 'maclean1' succeeded
CRS-2677: Stop of 'ora.asm' on 'maclean1' succeeded
CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on 'maclean1'
CRS-2677: Stop of 'ora.ctssd' on 'maclean1' succeeded
CRS-2677: Stop of 'ora.cluster_interconnect.haip' on 'maclean1' succeeded
CRS-2673: Attempting to stop 'ora.cssd' on 'maclean1'
CRS-2677: Stop of 'ora.cssd' on 'maclean1' succeeded
CRS-2673: Attempting to stop 'ora.gipcd' on 'maclean1'
CRS-2677: Stop of 'ora.gipcd' on 'maclean1' succeeded
CRS-2673: Attempting to stop 'ora.gpnpd' on 'maclean1'
CRS-2677: Stop of 'ora.gpnpd' on 'maclean1' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'maclean1' has completed
CRS-4133: Oracle High Availability Services has been stopped.

[root@maclean1 ~]# crsctl stat res -t
--------------------------------------------------------------------------------
NAME           TARGET  STATE        SERVER                   STATE_DETAILS       
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.BACKUPDG.dg
               ONLINE  ONLINE       maclean1                                     
ora.DATA.dg
               ONLINE  ONLINE       maclean1                                     
ora.LISTENER.lsnr
               ONLINE  ONLINE       maclean1                                     
ora.SYSTEMDG.dg
               ONLINE  ONLINE       maclean1                                     
ora.asm
               ONLINE  ONLINE       maclean1                 Started             
ora.gsd
               OFFLINE OFFLINE      maclean1                                     
ora.net1.network
               ONLINE  ONLINE       maclean1                                     
ora.ons
               ONLINE  ONLINE       maclean1                                     
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.LISTENER_SCAN1.lsnr
      1        ONLINE  ONLINE       maclean1                                     
ora.cvu
      1        ONLINE  ONLINE       maclean1                                     
ora.maclean1.vip
      1        ONLINE  ONLINE       maclean1                                     
ora.maclean2.vip
      1        ONLINE  INTERMEDIATE maclean1                 FAILED OVER         
ora.oc4j
      1        ONLINE  OFFLINE                               STARTING            
ora.prod.db
      1        ONLINE  OFFLINE                               Instance Shutdown,S 
                                                             TARTING             
      2        ONLINE  OFFLINE                                                   
ora.scan1.vip
      1        ONLINE  ONLINE       maclean1

 

 

因为上面更新了ASM共享使用的SPFILE，所以其他节点上一般不会存在问题，直接重启后CRS即可正常工作。

 

由于11gR2 RAC+ASM的启动依赖较为复杂，虽然通过以上操作但你仍可能在修改ASM DISK PATH路径时遇到一些问题， 若无法解决请去该问题的原帖提问，谢谢配合！

 

Posted

in

by

mac

Tags:

Comments

8 responses to “在11gR2 RAC中修改ASM DISK Path磁盘路径”

  1. kim_meng Avatar
    kim_meng

    grid启动服务顺序比较复杂，ML可以给讲解一下吗？
    grid一旦出现异常，如何定位，查看那些log文件，具体顺序如何？

    Reply
  2. ylw6006 Avatar
    ylw6006

    很好的文章，赞一个！
    另外想请教下刘总，在10g里面我可以用srvct命令先关闭数据库，再关闭asm，最后关闭nodeapps，启动的顺序正好相反；那么在11g rac中，如何实现同样的效果呢？一定要用crsctl stop resource -all命令吗？

    Reply
  3. 针对11.2 RAC丢失OCR和Votedisk所在ASM Diskgroup的恢复手段 | Ask Maclean Oracle Blog

    […] diskgroup导致11gR2 RAC cluster无法正常启动的问题，最早我在《在11gR2 RAC中修改ASM DISK Path磁盘路径》一文中介绍了，如何在不启动CRS的情况下启动11.2中的ASM实例并实施操作， […]

    Reply
  4. 深入了解Oracle ASM(一):基础概念 | Ask Maclean Oracle Blog

    […] 在11gR2 RAC中修改ASM DISK Path磁盘路径 […]

    Reply
  5. 0racle 11.2.0.3 RAC在线迁移OCR&VOTE ASM磁盘组 | Focus on Oracle

    […] 以上为Oracle 11gR2在线迁移OCR&VOTE ASM磁盘组过程，其中不涉及存储中所使用多路径软件问题，如果涉及到存储磁盘路径改变，则需要去修改ASM实例的disk_strings参数，具体操作可以参考Maclea的这篇博客在11gR2 RAC中修改ASM DISK Path磁盘路径，论坛贴链接[讨论]11G RAC更换存储，如有问题，欢迎讨论。 […]

    Reply
  6. 深入了解Oracle ASM(二):基础概念 – Oracle数据库数据恢复、性能优化来问问AskMaclean

    […] 在11gR2 RAC中修改ASM DISK Path磁盘路径 […]

    Reply
  7. donnie Avatar
    donnie

    求助，关于 oracle 11gr2 版本为：11.2.0.4，在安装RAC+mutipath+asm安装到执行该脚本时：/u01/app/11.2.0/grid/root.sh出现如下的错误：hanging permissions of /u01/app/grid/oraInventory.Adding read,write permissions for group.Removing read,write,execute permissions for world.Changing groupname of /u01/app/grid/oraInventory to oinstall.The execution of the script is complete.[root@kadrac1 grid]# /u01/app/11.2.0/grid/root.shPerforming root user operation for Oracle 11g The following environment variables are set as: ORACLE_OWNER= grid ORACLE_HOME= /u01/app/11.2.0/gridEnter the full pathname of the local bin directory: [/usr/local/bin]: The contents of “dbhome” have not changed. No need to overwrite.The contents of “oraenv” have not changed. No need to overwrite.The contents of “coraenv” have not changed. No need to overwrite.Creating /etc/oratab file…Entries will be added to the /etc/oratab file as needed byDatabase Configuration Assistant when a database is createdFinished running generic part of root script.Now product-specific root actions will be performed.Using configuration parameter file: /u01/app/11.2.0/grid/crs/install/crsconfig_paramsCreating trace directoryUser ignored Prerequisites during installationInstalling Trace File AnalyzerOLR initialization – successful root wallet root wallet cert root cert export peer wallet profile reader wallet pa wallet peer wallet keys pa wallet keys peer cert request pa cert request peer cert pa cert peer root cert TP profile reader root cert TP pa root cert TP peer pa cert TP pa peer cert TP profile reader pa cert TP profile reader peer cert TP peer user cert pa user certAdding Clusterware entries to inittabCRS-2672: Attempting to start ‘ora.mdnsd’ on ‘kadrac1’CRS-2676: Start of ‘ora.mdnsd’ on ‘kadrac1’ succeededCRS-2672: Attempting to start ‘ora.gpnpd’ on ‘kadrac1’CRS-2676: Start of ‘ora.gpnpd’ on ‘kadrac1’ succeededCRS-2672: Attempting to start ‘ora.cssdmonitor’ on ‘kadrac1’CRS-2672: Attempting to start ‘ora.gipcd’ on ‘kadrac1’CRS-2676: Start of ‘ora.cssdmonitor’ on ‘kadrac1’ succeededCRS-2676: Start of ‘ora.gipcd’ on ‘kadrac1’ succeededCRS-2672: Attempting to start ‘ora.cssd’ on ‘kadrac1’CRS-2672: Attempting to start ‘ora.diskmon’ on ‘kadrac1’CRS-2676: Start of ‘ora.diskmon’ on ‘kadrac1’ succeededCRS-2676: Start of ‘ora.cssd’ on ‘kadrac1’ succeededASM created and started successfully.Disk Group OCR_VOTE created successfully.The ora.asm resource is not ONLINEDid not succssfully configure and start ASM at /u01/app/11.2.0/grid/crs/install/crsconfig_lib.pm line 6912./u01/app/11.2.0/grid/perl/bin/perl -I/u01/app/11.2.0/grid/perl/lib -I/u01/app/11.2.0/grid/crs/install /u01/app/11.2.0/grid/crs/install/rootcrs.pl execution failed不知道如何解决，这个问题已经困扰我两天了，在此请教各位了，本人在此感谢。

    Reply
  8. 0racle 11.2.0.3 RAC在线迁移OCR&VOTE ASM磁盘组 – Focus on Oracle

    […] 以上为Oracle 11gR2在线迁移OCR&VOTE ASM磁盘组过程，其中不涉及存储中所使用多路径软件问题，如果涉及到存储磁盘路径改变，则需要去修改ASM实例的disk_strings参数，具体操作可以参考Maclea的这篇博客在11gR2 RAC中修改ASM DISK Path磁盘路径，论坛贴链接[讨论]11G RAC更换存储，如有问题，欢迎讨论。 […]

    Reply

Leave a Reply

Your email address will not be published. Required fields are marked *