StartupForce

Online Disk Migration through ASM Rebalancing

September 5, 2013 Bharat Leave a comment

Rebalancing feature of ASM is very helpful whenever you want to change the underlying disks aka storage without taking the database offline

ASM Rebalancing:

It automatically rebalances—redistributes file data evenly across all the disks of the disk group—after you add disks to or drop disks from a disk group. ASM automatically rebalances whenever disks are added or dropped. For a normal drop operation (without the FORCE option), a disk is not released from a disk group until data is moved off of the disk through rebalancing.

Likewise, a newly added disk cannot support its share of the I/O workload until rebalancing completes. It is more efficient to add or drop multiple disks at the same time so that they are rebalanced as a single operation. This avoids unnecessary movement of data.

The Setup:

There are 3 diskgroups DATA, FRA & OCR_VD with one disk each.

SQL>  select a.name "Group" ,b.name "Disk",b.path "Path",
b.MODE_STATUS "Status" from v$asm_diskgroup a,v$asm_disk b 
where a.GROUP_NUMBER=b.GROUP_NUMBER;

Group      Disk                 Path          Status
---------- -------------  ------------------ -------
DATA       DATA_0000      /dev/asm-disk2      ONLINE
FRA        FRA_0000       /dev/asm-disk3      ONLINE
OCR_VD     OCR_VD_0000    /dev/asm-disk1      ONLINE

We will add new disks to the RAC instances, in DATA diskgroup and wait for the rebalancing to complete. Once the rebalancing is done the old disks can be removed from the diskgroups and another rebalancing will happen. Once completed, old disks can be removed

Available ASM disks at OS level.

Disks are already partitioned at OS level and are ready for addition.

 [grid@rac1 ~]$ ls -lrt /dev/asm-disk*
 brw-rw---- 1 grid oinstall 8, 23 Aug 28 22:30 /dev/asm-disk3
 brw-rw---- 1 grid oinstall 8, 22 Aug 28 22:46 /dev/asm-disk2
 brw-rw---- 1 grid oinstall 8, 21 Aug 28 22:46 /dev/asm-disk1
 brw-rw---- 1 grid oinstall 8, 26 Aug 29  2013 /dev/asm-disk6
 brw-rw---- 1 grid oinstall 8, 24 Aug 29  2013 /dev/asm-disk4
 brw-rw---- 1 grid oinstall 8, 27 Aug 29  2013 /dev/asm-disk7
 brw-rw---- 1 grid oinstall 8, 29 Aug 29  2013 /dev/asm-disk9
 brw-rw---- 1 grid oinstall 8, 30 Aug 29  2013 /dev/asm-disk10
 brw-rw---- 1 grid oinstall 8, 25 Aug 29  2013 /dev/asm-disk5
 brw-rw---- 1 grid oinstall 8, 28 Aug 29  2013 /dev/asm-disk8
 brw-rw---- 1 grid oinstall 8, 31 Aug 29  2013 /dev/asm-disk11
SQL> sho parameter asm_diskstring
NAME                                 TYPE        VALUE
 ------------------------------------ ----------- ------------------------------
 asm_diskstring                       string      /dev/asm-disk*

These new disks are visible in the v$asm_disk view, but check for header status

SQL> select path, header_status, mode_status from v$asm_disk;
PATH            HEADER_STATU MODE_ST
 --------------- ------------ -------
 /dev/asm-disk11 CANDIDATE    ONLINE
 /dev/asm-disk8  CANDIDATE    ONLINE
 /dev/asm-disk5  CANDIDATE    ONLINE
 /dev/asm-disk10 CANDIDATE    ONLINE
 /dev/asm-disk9  CANDIDATE    ONLINE
 /dev/asm-disk7  CANDIDATE    ONLINE
 /dev/asm-disk6  CANDIDATE    ONLINE
 /dev/asm-disk4  CANDIDATE    ONLINE
 /dev/asm-disk2  MEMBER       ONLINE
 /dev/asm-disk1  MEMBER       ONLINE
 /dev/asm-disk3  MEMBER       ONLINE
11 rows selected.

Before adding the disks check for datafile location and observe that the database is online,

SQL> select file_name from dba_data_files;
FILE_NAME
 ---------------------------------------------
 +DATA/racdb/datafile/undotbs2.268.823906247
 +DATA/racdb/datafile/system.260.823906197
 +DATA/racdb/datafile/sysaux.265.823906229
 +DATA/racdb/datafile/undotbs1.264.823906223
 +DATA/racdb/datafile/users.267.823906245
SQL> select name from v$controlfile;
NAME
 -------------------------------------------
 +DATA/racdb/datafile/control01.ctl
 +DATA/racdb/datafile/control02.ctl
SQL> select member from v$logfile;
MEMBER
 ---------------------------------------------
 +DATA/racdb/onlinelog/group_1.259.823905645
 +DATA/racdb/onlinelog/group_2.263.823905355
 +DATA/racdb/onlinelog/group_3.262.823905373
 +DATA/racdb/onlinelog/group_4.261.823905379

Addition of new disk in DATA diskgroup

From ASM instance,

SQL> alter diskgroup data add disk '/dev/asm-disk4','/dev/asm-disk5' 
                                                    rebalance power 5;
Diskgroup altered.

This will immediately start a rebalancing operation that we can view through the v$asm_operation view.

SQL> select group_number, operation, est_minutes from v$asm_operation;
GROUP_NUMBER OPERA EST_MINUTES
 ------------ ----- -----------
 1 REBAL           4

Once completed the operation will disappear

SQL>  select group_number, operation, est_minutes from v$asm_operation;
no rows selected

Check for the addition of the disks:

SQL> select a.name "Group" ,b.name "Disk",b.path "Path",
b.MODE_STATUS "Status" from v$asm_diskgroup a,v$asm_disk b 
where a.GROUP_NUMBER=b.GROUP_NUMBER and a.name='DATA';
Group Disk       Path                 Status
 ----- ---------- -------------------- -------
 DATA  DATA_0000  /dev/asm-disk2       ONLINE
 DATA  DATA_0002  /dev/asm-disk5       ONLINE
 DATA  DATA_0001  /dev/asm-disk4       ONLINE

Drop the old disks now

It will automatically rebalancing where ASM relocates all the data from the disks being dropped. When the data is moved away the disk header is cleared and the disk will no longer part of the diskgroup.

SQL> alter diskgroup data drop disk DATA_0001,DATA_0002 rebalance power 4;
Diskgroup altered.

SQL> select group_number, operation, est_minutes from v$asm_operation;
GROUP_NUMBER OPERA EST_MINUTES
 ------------ ----- -----------
 1 REBAL           3
SQL> select group_number, operation, est_minutes from v$asm_operation;
no rows selected

If you will query v$asm_disk will show that asm1 and asm2 disks as FORMER, means they were cleanly dropped and it is now safe to remove them from the storage.

SQL> select path, header_status, mode_status from v$asm_disk;
PATH                 HEADER_STATU MODE_ST
 -------------------- ------------ -------
 /dev/asm-disk5       FORMER       ONLINE
 /dev/asm-disk4       FORMER       ONLINE
 /dev/asm-disk11      CANDIDATE    ONLINE
 /dev/asm-disk8       CANDIDATE    ONLINE
 /dev/asm-disk10      CANDIDATE    ONLINE
 /dev/asm-disk9       CANDIDATE    ONLINE
 /dev/asm-disk6       CANDIDATE    ONLINE
 /dev/asm-disk7       CANDIDATE    ONLINE
 /dev/asm-disk2       MEMBER       ONLINE
 /dev/asm-disk1       MEMBER       ONLINE
 /dev/asm-disk3       MEMBER       ONLINE

ASM disk names, group membership and status after dropping

SQL> select a.name "Group" ,b.name "Disk",b.path "Path",
b.MODE_STATUS "Status" 
from v$asm_diskgroup a,v$asm_disk b 
where a.GROUP_NUMBER=b.GROUP_NUMBER and a.name='DATA';
Group                          Disk                Path               Status
 ------------------------------ ---------------- -------------------- -------
 DATA                           DATA_0000        /dev/asm-disk2       ONLINE

Note: On RAC environment, “ASM can perform one disk group rebalance at a time on a given instance. Therefore, if you have initiated multiple rebalances on different disk groups, then Oracle processes this operation serially. However, you can initiate rebalances on different disk groups on different nodes in parallel”

Categories: RAC Tags: ASM, rebalance

What are candidate & provisioned disks in ASM

August 30, 2013 Bharat Leave a comment

Disks that were discovered but that have not yet been assigned to a disk group have a header status of either CANDIDATE or PROVISIONED.

CANDIDATE

Disk is not part of a disk group and may be added to a disk group with the ALTER DISKGROUP statement

PROVISIONED

Disk is not part of a disk group and may be added to a disk group with the ALTER DISKGROUP statement. The PROVISIONED header status is different from the CANDIDATE header status. Here PROVISIONED implies that an additional platform-specific action has been taken by an administrator to make the disk available for Automatic Storage Management.

For example, on Windows, the administrator used asmtool or asmtoolg to stamp the disk with a header, or on Linux, the administrator used ASMLib to prepare the disk for ASM.

Below are the various HEADER_STATUS in the v$ASM_DISK. I have taken below status from 11gR2.

· UNKNOWN – Automatic Storage Management disk header has not been read

· CANDIDATE – Disk is not part of a disk group and may be added to a disk group with the ALTER DISKGROUP statement

· INCOMPATIBLE – Version number in the disk header is not compatible with the Automatic Storage Management software version.

· PROVISIONED – Disk is not part of a disk group and may be added to a disk group with the ALTER DISKGROUP statement. The PROVISIONED header status is different from the CANDIDATE header status in that PROVISIONED implies that an additional platform-specific action has been taken by an administrator to make the disk available for Automatic Storage Management.

· MEMBER – Disk is a member of an existing disk group. No attempt should be made to add the disk to a different disk group. The ALTER DISKGROUP statement will reject such an addition unless overridden with the FORCE option

· FORMER – Disk was once part of a disk group but has been dropped cleanly from the group. It may be added to a new disk group with the ALTER DISKGROUP statement.

· CONFLICT – Automatic Storage Management disk was not mounted due to a conflict

· FOREIGN – Disk contains data created by an Oracle product other than ASM. This includes datafiles, logfiles, and OCR disks.

When adding a disk, the FORCE option must be used if Oracle ASM recognizes that the disk was managed by Oracle. Such a disk appears

in the V$ASM_DISK view with a status of FOREIGN.