Friday, 31 May 2013

RMAN has deleted backupset but not freed up space to OS


I am having strange issue in our production system. And it caused to fail rman subsequent backup. 
The issue was previous deleted rman backupset file still being access by other process. And to resolve this issue lsof +L1 become boon to me. 

Here I have resolve the issue.

[root@PRODB01 backupset]#
[root@PRODB01 backupset]# df -h /flash_recovery_area/
Filesystem            Size  Used Avail Use% Mounted on
                      1.5T  1.1T  343G  77% /flash_recovery_area

[root@PRODB01 backupset]# cd /flash_recovery_area/
data_pump/  PRODDB/   lost+found/

[root@PRODB01 backupset]# cd /flash_recovery_area/

[root@PRODB01 flash_recovery_area]# du -sh *
337G    data_pump
386G    PRODDB
16K     lost+found

[root@PRODB01 flash_recovery_area]# lsof +L1 /flash_recovery_area | grep backupset
rsync     25269    root    3r   REG 253,10 396905644032     0  83329027 /flash_recovery_area/PRODDB/backupset/2013_05_28/o1_mf_nnndf_BACKUP_LIVEDB_8t9txcvw_.bkp (deleted)

[root@PRODB01 flash_recovery_area]#
[root@PRODB01 flash_recovery_area]# ps -ef | grep 25269
root      8980 14727  0 11:55 pts/6    00:00:00 grep 25269
root     25269 25242  0 May28 ?        00:27:58 rsync --verbose --archive --hard-links --force --delete --stats -progress --links --times --recursive --relative --exclude /proc --exclude /dev --exclude /mnt --exclude /rman2 --exclude /a --exclude /vol --exclude /tmp --exclude /rman --exclude /willow \ --exclude /var/spool/lpd --delete-excluded --exclude core --ignore-errors --partial --backup --backup-dir=/scorpio/scorpio9/PRODB01/28-05-13 -e ssh -v / scorpio:/scorpio/scorpio9/PRODB01/current
root     25277 25269  3 May28 ?        02:26:35 ssh scorpio rsync --server --backup-dir /scorpio/scorpio9/PRODB01/28-05-13 --delete-excluded --force --partial --ignore-errors . /scorpio/scorpio9/PRODB01/current

[root@PRODB01 flash_recovery_area]# kill -9 25269
[root@PRODB01 flash_recovery_area]# df -h /flash_recovery_area/
 1.5T  744G  691G  52% /3parrman

Job Done ! Space revert it back !

No comments: