Read ahead sectors 导致rman 备份慢 DBA无解！

Read ahead sectors     auto

APPLIES TO:

Oracle Database - Enterprise Edition - Version 11.2.0.4 and later
Information in this document applies to any platform.

SYMPTOMS

RMAN backup to disk through the following command takes 2 days to complete:
backup as backupset incremental level 0 section size 32g database plus archivelog section size 32g DELETE ALL INPUT;
 

While on another server with same OS and RDBMS version and similar hardwares (even less CPUs), 
RMAN backup takes 6 hours to complete. 

Part of sample script:
 

run {
resync catalog;
sql 'alter system set "_backup_disk_bufcnt"=64 scope=memory';
sql 'alter system set "_backup_disk_bufsz"=1048576 scope=memory';
allocate channel ch01 device type disk;
allocate channel ch02 device type disk;
backup as backupset incremental level 0 section size 32g format '<file system location 1>/full.%d.%s.%p' database plus archivelog section size 32g format '<file system location 2>/arch.%d.%s.%p' DELETE ALL INPUT;
release channel ch01;
release channel ch02;
}

Adding the VALIDATE option to the RMAN command according to note RMAN Performance Troubleshooting (Doc ID 1326686.1),
and with 4 disk channels allocated makes not much difference in performance.

CHANGES

CAUSE

The cause is that the logical volume where the datafiles reside has the 'Read ahead sectors' attribute set to 0.

And this is not the case on the server without the performance problem.  

The bottleneck is the datafile read speed which extremely impacts the backup.

SOLUTION

Modify the 'read ahead sectors' attribute to a non-zero value.  In this case, after setting the value to 8192 the
backup performance drastically improves.

GOAL

For performance reasons instead of using one large LVM volume it might be replaced by multiple small striped (RAID0) volumes.

There is no command to directly transform a single disk logical volume in a striped logical volume.

This document will explain the procedure to perform the conversion.
 

SOLUTION

Before start take a full backup of the volume or filesystem.

This procedure can be performed in a live system, without the need to reboot.

The procedure consist in create a mirror between the original volume and the new striped volumes, after remove the original volume from the mirror leaving only the striped volumes.

To be able to perform the conversion, the number of logical extents (LE) in the LVM logical volume (LV) must be divisible by the number of physical volumes (PV) that will be used in the future striped LV.

Check the number of assigned LE:

# lvdisplay
 --- Logical volume ---
 LV Path                /dev/<VOLUME_GROUP>/<LOGICAL_VOLUME>
 LV Name                <LOGICAL_VOLUME>
 VG Name                <VOLUME_GROUP>
 LV UUID                <VOLUME_UUID>
 LV Write Access        read/write
 LV Creation host, time <HOSTNAME>, 2020-12-10 08:38:44 +0000
 LV Status              available
 # open                 1
 LV Size                <10.00 GiB
 Current LE             2559
 Segments               1
 Allocation             inherit
 Read ahead sectors     auto
 - currently set to     256
 Block device           252:0

In this case the number of LE is 2559.

To know the number of needed LE, you can use:
le = number of assigned LE, from previous command (2559 in this case)
pv = number of striped PV to be used in future (ex: 5)

$ le=2559; pv=5; echo $((le%pv?(le/pv+1)*pv:le))
2560

In this case one extra LE is needed. Example to extend the LV:

#  lvresize --extents +1 /dev/<VOLUME_GROUP>/<LOGICAL_VOLUME> /dev/sdb
 Size of logical volume <VOLUME_GROUP>/<LOGICAL_VOLUME> changed from <10 GiB (2559 extents) to <10 giB (2560 extents).
 Logical volume <VOLUME_GROUP>/<LOGICAL_VOLUME> successfully resized.

After the LV is prepared proceed:

1. Add the new PVs to the LV creating a striped mirror. The number of stripes should be the number of PVs:

   # lvconvert --type mirror -m 1 --mirrorlog core --stripes 2 /dev/<VOLUME_GROUP>/<LOGICAL_VOLUME> /dev/<PV1> /dev/<PV2> ... /dev/<PV5>

   
    
2. Remove the initial PV from the LV:

   # lvconvert -m0 <VOLUME_GROUP>/<LOGICAL_VOLUME> /dev/<INITIAL_PV>

   
    
3. Remove the PV from LVM:

   # vgreduce <VOLUME_GROUP> /dev/<INITIAL_PV>
   # pvremove /dev/<INITIAL_PV>