Ask Maclean

ASM Metadata元数据介绍

1.    ASM Metadata介绍

 

 

ASM基础概念：

 

  • ASM的最小存储单位是一个”allocation unit”(AU)，通常为1MB，在Exadata上推荐为4MB
  • ASM的核心是存储文件
  • 文件被划分为多个文件片，称之为”extent”
  • 11g之前extent的大小总是为一个AU，11g之后一个extent可以是1 or 8 or 64个AU
  • ASM使用file extent map维护文件extent的位置
  • ASM在LUN DISK的头部header维护其元数据，而非数据字典
  • 同时RDBMS DB会在shared pool中缓存file extent map，当server process处理IO时使用
  • 因为ASM instance使用类似于普通RDBMS的原理的instance/crash recovery，所以ASM instance奔溃后总是能复原的

 

ASM将任何文件以AU大小均匀分布在Disk Group的所有Disk上。每一个ASM Disk均被维护以保持同样的使用比率。这保证同一个Disk Group中的所有Disk的IO负载基本一致。由于ASM在一个Disk Group中的磁盘上的负载均衡，所以为同一个物理磁盘的不同区域划分为2个ASM Disk不会对性能有所影响；而同一个物理磁盘上划分2个不同分区置于不同的2个Disk Group则有效。

当ASM Disk Group启用冗余时单个ASM Disk仅是一个失败单元。对于该ASM Disk的写失败在10g会自动从该Disk Group drop掉该Disk，前提是该Disk的丢失被容许。

 

Allocation Unit

每一个ASM Disk都被划分为许多个AU allocation units(单个AU 的大小在 1MB ~64MB，注意总是2的次方MB)。而且AU allocation unit也是Disk Group的基本分配单元。一个ASM Disk上的可用空间总是整数倍个AU。在每一个ASM Disk的头部均有一个表，该表的每一条记录代表该ASM Disk上的一个AU。文件的extent指针(pointer)给出了ASM Disk Number磁盘号和AU号，这就描述了该extent的物理位置。由于所有的空间操作都以AU为单位，所以不存在所谓ASM碎片这样的概念和问题。

一个AU(1M~64M)足够小，以便一个文件总是要包含很多个AU，这样就可以分布在很多磁盘上，也不会造成热点。一个AU又足够大以便能够在一个IO操作中访问它，以获得更加的吞吐量，也能提供高效的顺序访问。访问一个AU的时间将更多的消耗在磁盘传输速率上而非花在寻找AU头上。对于Disk Group的重新平衡也是对每一个AU逐次做的。

 

 

asm disk的前50个AU(50MB)是为asm metadata保留的

ASM 的前255个file number是为metadata file保留的，文件号从1开始， file numner=1的1号文件为ASM的file directory;普通的ASM File的file number从256开始

 

 

可以使用脚本 GetAsmDH.sh来定期备份 ASM header的metadata数据， GetAsmDH.sh本质是使用 dd 来备份asm header 前1MB的数据

GetAsmDH.sh:

 

 

#!/bin/sh

mkdir /tmp/HC 2> /dev/null
rm -f /tmp/HC/asmdisks.lst 2> /dev/null
rm -f /tmp/HC/asm_diskh.sh 2> /dev/null

echo " "
echo "############################################"
echo " 1) Collecting Information About the Disks:"
echo "############################################"

sqlplus '/nolog' <<eof
connect / as sysdba
set linesize 90
col path format a60
set heading off
set head off
set feedback off
spool /tmp/HC/asmdisks.lst
select group_number,disk_number,path from v\$asm_disk_stat where group_number > 0 order by group_number,disk_number;
spool off;
eof
echo " "
echo " "

ls -l /tmp/HC/asmdisks.lst


echo " "
echo "############################################"
echo " 2) Generating "asm_diskh.sh" script."
echo "############################################"
echo " "

grep -v SQL /tmp/HC/asmdisks.lst > /tmp/HC/asmdisks_tmp.lst

mv /tmp/HC/asmdisks_tmp.lst /tmp/HC/asmdisks.lst 

sed 's/ORCL:/\/dev\/oracleasm\/disks\//g' </tmp/HC/asmdisks.lst>/tmp/HC/asmdisks_NEW.lst

mv /tmp/HC/asmdisks_NEW.lst /tmp/HC/asmdisks.lst



cat /tmp/HC/asmdisks.lst|while read LINE
do
comm=`echo $LINE|awk '{print "dd if="$3 " of=/tmp/HC/dsk_"$1"_"$2".dd bs=1048576 count=1"}'`
echo $comm >> /tmp/HC/asm_diskh.sh
done 

chmod 700 /tmp/HC/asm_diskh.sh

ls -l /tmp/HC/asm_diskh.sh

echo " "
echo "############################################"
echo " 3) Executing asm_diskh.sh script to "
echo " generate dd dumps."
echo "############################################"
echo " "


### For display only
/tmp/HC/asm_diskh.sh 2> /dev/null
ls -l /tmp/HC/*dd

echo " "
echo "############################################"
echo " 4) Compressing dd dumps in the next format:"
echo " (asm_dd_header_all_<date_time>.tar)"
echo "############################################"
echo " "




NOW=$(date +"%m-%d-%Y_%T")

tar -cvf /tmp/HC/asm_dd_header_all_$NOW.tar /tmp/HC/*.dd 2> /dev/null

compress /tmp/HC/asm_dd_header_all_$NOW.tar

ls -l /tmp/HC/*.Z

当需要恢复 ASM header时可以直接 将备份的文件dd回去 ，但是要注意dd加上conv=notrunc
其他一些有用的脚本：

 

以下脚本用于ASM修复disk header时：
 
 
1. dd各种有用的metadata block :
 
#! /bin/sh
rm /tmp/kfed_DH.out /tmp/kfed_FS.out /tmp/kfed_BK.out /tmp/kfed_FD.out /tmp/kfed_DD.out /tmp/kfed_PST.out
for i in `ls /dev/asm-disk*`
do
echo $i >> /tmp/kfed_DH.out
kfed read $i >> /tmp/kfed_DH.out
echo $i >> /tmp/kfed_FS.out
kfed read $i blkn=1 >> /tmp/kfed_FS.out
echo $i >> /tmp/kfed_BK.out
kfed read $i aun=1 blkn=254 >> /tmp/kfed_BK.out
echo $i >> /tmp/kfed_FD.out
kfed read $i aun=2 blkn=1 >> /tmp/kfed_FD.out
echo $i >> /tmp/kfed_DD.out
kfed read $i aun=2 blkn=2 >> /tmp/kfed_DD.out
echo $i >> /tmp/kfed_PST.out
kfed read $i aun=1 blkn=2 >> /tmp/kfed_PST.out
done
 
 
 
kfed_DH.out ==>KFBTYP_DISKHEAD      aun=0 blkn=0
kfed_FS.out ==>  KFBTYP_FREESPC      aun=1 blkn=0
kfed_BK.out  ==> KFBTYP_DISKHEAD DISK HEAD BACKUP   aun=1 blkn=254
kfed_FD.out  ==> KFBTYP_FILEDIR   aun=2  blkn=1
kfed_DD.out  ==> KFBTYP_FILEDIR  aun=2 blkn=2
kfed_PST.out ==> KFBTYP_PST_NONE aun=1 blkn=2
 
2 . Query ASM header from SQL:
 
 
spool asm_info.html
set pagesize 1000
set linesize 250
set feedback off
col bytes format 999,999,999,999
col space format 999,999,999,999
col gn format 999
col name format a20
col au format 99999999
col state format a12
col type format a12
col total_mb format 999,999,999
col free_mb format 999,999,999
col od format 999
col compatibility format a12
col dn format 999
col mount_status format a12
col header_status format a12
col mode_status format a12
col mode format a12
col failgroup format a20
col label format a12
col path format a45
col path1 format a40
col path2 format a40
col path3 format a40
col bytes_read format 999,999,999,999,999
col bytes_written format 999,999,999,999,999
col cold_bytes_read format 999,999,999,999,999
col cold_bytes_written format 999,999,999,999,999

alter session set nls_date_format='DD-MON-YYYY HH24:MI:SS' ;

select to_char(sysdate, 'DD-MON-YYYY HH24:MI:SS' ) current_time from dual;
select group_number gn, name, allocation_unit_size au, state, type, total_mb, free_mb, offline_disks od, compatibility from v$asm_diskgroup;
select group_number gn,disk_number dn, mount_status, header_status,mode_status,state, total_mb, free_mb,name, failgroup, label, path,create_date, mount_date from v$asm_disk order by group_number, disk_number;

break on g_n skip 1
break on failgroup skip 1
compute sum of t_mb f_mb on failgroup
compute count of failgroup on failgroup

select g.group_number g_n,g.disk_number d_n,g.name , g.path , g.total_mb t_mb,g.free_mb f_mb,g.failgroup from v$asm_disk g order by g_n, failgroup, d_n;
SET MARKUP HTML ON
set echo on
select 'THIS ASM REPORT WAS GENERATED AT: ==)> ' , sysdate " " from dual;
select 'HOSTNAME ASSOCIATED WITH THIS ASM INSTANCE: ==)> ' , MACHINE " " from v$session where program like '%SMON%';
select * from v$asm_diskgroup;
SELECT * FROM V$ASM_DISK ORDER BY GROUP_NUMBER,DISK_NUMBER;
SELECT * FROM V$ASM_CLIENT;
select * from V$ASM_ATTRIBUTE;
select * from v$asm_operation;
select * from v$version;
show parameter
show sga
spool off
exit
 
 
 
AMDU result:
 
 
Placeholder for AMDU binaries and using with ASM 10g (Doc ID 553639.1)

amdu -diskstring '/dev/asm-disk*' -dump 'MACLEAN_DG' -noimage
 
 
 
4. 脚本查找LISTHEAD
 
 
#!/bin/bash
# Usage: scan.sh     
i=0
size=0
asize=$2
rm list.txt
echo AUSZIE=$asize
while [ 1 ]
do
kfed read $1 ausz=$asize aunum=$i blknum=0 | grep LISTHEAD > list.txt
size=$(stat -c %s list.txt)
if [ $size -gt 0 ]; then
  echo LISTHEAD is found in AU=$i FILE=lhAU$i.txt
  kfed read $1 ausz=$asize aunum=$i blknum=0 text=lhAU$i.txt
fi
i=$[$i+1]
if [ $i -eq $3 ]; then
  echo $3 AUs scanned
  exit 0
fi
done
 
 
使用方法：
 
[grid@vmac1 tmp]$ ./scan.sh /dev/asm-diskb 1048576 10
AUSZIE=1048576
LISTHEAD is found in AU=2 FILE=lhAU2.txt
10 AUs scanned

Posted

in

by

mac

Tags:

Comments

Leave a Reply

Your email address will not be published. Required fields are marked *