The staleness directory (ASM file number 12) contains metadata to map the slots in the staleness registry to particular disks and ASM clients. The staleness registry (ASM file number 254) tracks allocation units that become stale while the disks are offline. This applies to normal and high redundancy disk groups with the attribute COMPATIBLE.RDBMS set to 11.1 or higher. The staleness metadata is created when needed, and grows to accommodate additional offline disks.
When a disk goes offline, each RDBMS instance gets a slot in the staleness registry for that disk. This slot has a bit for each allocation unit in the offline disk. When an RDBMS instance I/O write is targeted for an offline disk, that instance sets the corresponding bit in the staleness registry.
When a disk is brought back online, ASM copies the allocation units, that have the staleness registry bit set, from the mirrored extents. Because only allocation units that should have changed while the disk was offline are updated, bringing a disk online is more efficient then adding a disk if was dropped instead of just offlined.
No stale disks
The staleness metadata structures are created as needed, which means the staleness directory and registry do not exist when all disks are online.
g.group_number “Group#”,
d.disk_number “Disk#”,
d.name “Disk”,
d.mode_status “Disk status”
FROM v$asm_disk d, v$asm_diskgroup g
WHERE g.group_number=d.group_number and g.group_number<>0
ORDER BY 1, 2, 3;
Disk group Group# Disk# Disk Disk status
———— ———- ———- ———— ————
DATA 1 0 ASMDISK1 ONLINE
1 ASMDISK2 ONLINE
2 ASMDISK3 ONLINE
RECO 2 0 ASMDISK4 ONLINE
1 ASMDISK5 ONLINE
2 ASMDISK6 ONLINE
SQL> SELECT x.number_kffxp “File#”,
x.disk_kffxp “Disk#”,
x.xnum_kffxp “Extent”,
x.au_kffxp “AU”,
d.name “Disk name”
FROM x$kffxp x, v$asm_disk_stat d
WHERE x.group_kffxp=d.group_number
and x.disk_kffxp=d.disk_number
and x.number_kffxp in (12, 254)
ORDER BY 1, 2;
no rows selected
Stale disks
Staleness information will be created when a disk goes offline, but only when there are I/O writes intended for offline disks.
In the following example, I will offline the disk manually, with the ALTER DISKGROUP OFFLINE DISK command. But as far as stalenss metadata is concerned, it will be created irrespective of how and why a disk goes offline.
Diskgroup altered.
SQL> SELECT g.name “Disk group”,
g.group_number “Group#”,
d.disk_number “Disk#”,
d.name “Disk”,
d.mode_status “Disk status”
FROM v$asm_disk d, v$asm_diskgroup g
WHERE g.group_number=d.group_number and g.group_number=2
ORDER BY 1, 2, 3;
Disk group Group# Disk# Disk Disk status
———— ———- ———- ———— ————
RECO 2 0 ASMDISK4 ONLINE
1 ASMDISK5 ONLINE
2 ASMDISK6 OFFLINE
Database keeps writing to this disk group, and after a while we see the staleness directory and staleness registry created for this disk group
x.disk_kffxp “Disk#”,
x.xnum_kffxp “Extent”,
x.au_kffxp “AU”,
d.name “Disk name”
FROM x$kffxp x, v$asm_disk_stat d
WHERE x.group_kffxp=d.group_number
and x.disk_kffxp=d.disk_number
and d.group_number=2
and x.number_kffxp in (12, 254)
ORDER BY 1, 2;
File# Disk# Extent AU Disk name
———- ———- ———- ———- ——————————
12 0 0 86 ASMDISK4
1 0 101 ASMDISK5
2 0 4294967294 ASMDISK6
254 0 0 85 ASMDISK4
1 0 100 ASMDISK5
2 0 4294967294 ASMDISK6
Look inside
There is not much to see in the actual metadata. Even kfed struggles to recognise these types of metadata blocks 🙂
kfbh.endian: 1 ; 0x000: 0x01
kfbh.hard: 130 ; 0x001: 0x82
kfbh.type: 21 ; 0x002: *** Unknown Enum ***
…
kffdnd.bnode.incarn: 1 ; 0x000: A=1 NUMM=0x0
kffdnd.bnode.frlist.number: 4294967295 ; 0x004: 0xffffffff
kffdnd.bnode.frlist.incarn: 0 ; 0x008: A=0 NUMM=0x0
kffdnd.overfl.number: 4294967295 ; 0x00c: 0xffffffff
kffdnd.overfl.incarn: 0 ; 0x010: A=0 NUMM=0x0
kffdnd.parent.number: 0 ; 0x014: 0x00000000
kffdnd.parent.incarn: 1 ; 0x018: A=1 NUMM=0x0
kffdnd.fstblk.number: 0 ; 0x01c: 0x00000000
kffdnd.fstblk.incarn: 1 ; 0x020: A=1 NUMM=0x0
kfdsde.entry.incarn: 1 ; 0x024: A=1 NUMM=0x0
kfdsde.entry.hash: 0 ; 0x028: 0x00000000
kfdsde.entry.refer.number: 4294967295 ; 0x02c: 0xffffffff
kfdsde.entry.refer.incarn: 0 ; 0x030: A=0 NUMM=0x0
kfdsde.cid: +ASMR ; 0x034: length=5
kfdsde.indlen: 1 ; 0x074: 0x0001
kfdsde.flags: 0 ; 0x076: 0x0000
kfdsde.spare1: 0 ; 0x078: 0x00000000
kfdsde.spare2: 0 ; 0x07c: 0x00000000
kfdsde.indices[0]: 0 ; 0x080: 0x00000000
kfdsde.indices[1]: 0 ; 0x084: 0x00000000
kfdsde.indices[2]: 0 ; 0x088: 0x00000000
…
$ kfed read /dev/oracleasm/disks/ASMDISK4 aun=85 | more
kfbh.endian: 1 ; 0x000: 0x01
kfbh.hard: 130 ; 0x001: 0x82
kfbh.type: 20 ; 0x002: *** Unknown Enum ***
…
kfdsHdrB.clientId: 1297301881 ; 0x000: 0x4d534179
kfdsHdrB.incarn: 0 ; 0x004: 0x00000000
kfdsHdrB.dskNum: 2 ; 0x008: 0x0002
kfdsHdrB.ub2spare: 0 ; 0x00a: 0x0000
ub1[0]: 0 ; 0x00c: 0x00
ub1[1]: 0 ; 0x00d: 0x00
ub1[2]: 0 ; 0x00e: 0x00
ub1[3]: 0 ; 0x00f: 0x00
ub1[4]: 0 ; 0x010: 0x00
ub1[5]: 0 ; 0x011: 0x00
ub1[6]: 0 ; 0x012: 0x00
ub1[7]: 16 ; 0x013: 0x10
ub1[8]: 0 ; 0x014: 0x00
…
Not much to see, as these are just bitmaps.
Conclusion
The staleness directory and staleness registry are supporting metadata structure for the disk offline and fast resync feature introduced in ASM version 11. The staleness directory contains metadata to map the slots in the staleness registry to particular disks and ASM clients. The staleness registry tracks allocation units that become stale while the disks are offline. This feature is relevant to normal and high redundancy disk groups only.
Leave a Reply