HEMESH ORAWORLD

Friday, 11 December 2009

Grid Control components be automatically started and stopped upon host shutdown or reboot on Red Hat/Oracle Enterprise Linux

How can the Grid Control components be automatically started and stopped upon host shutdown or reboot on Red Hat/Oracle Enterprise Linux?

Applies to:

Enterprise Manager Grid Control - Version: 10.2.0.1 to 10.2.0.4
Linux x86

Goal

How can the Grid Control components be automatically started and stopped upon host shutdown or reboot on Red Hat/Oracle Enterprise Linux?

Solution

I. Automatically Starting Grid Control components on Red Hat/Oracle Enterprise Linux

Grid Control provides several scripts that allow for automatic starting of its components during
the boot process. The specific scripts for each component are located on its ORACLE_HOME. On a normal Grid Control installation the scripts controlling the automatic startup/shutdown are

$DB_HOME/install/unix/scripts/seedstup
$OMS_HOME/install/unix/scripts/agentstup
$AGENT_HOME/install/unix/scripts/agentstup

There is also a general startup script that will be run by the Linux boot up process. The script called gcstarup is located on
/etc/rc.d/init.d/gcstartup,
and to be called at startup two links are created to it
/etc/rc.d/rc3.d/S99gcstartup
/etc/rc.d/rc5.d/S99gcstartup

By default, this functionality should be activated after installation, however, sometimes there is some misconfiguration that prevents Grid Control components to start automatically on boot time. To check that the configuration is correct, follow the steps below.

1.- Database script.
1.1 If Grid Control has been installed with the new database option, the startup/shutdown script for the database should be located on the following location

$DB_HOME/install/unix/scripts/seedstup.

Check the following lines and ensure that the correct values are present

export ORACLE_HOME=<ORACLE_HOME_PATH>
user=<OS_user_that_owns_the_installation>
export ORACLE_SID=<SID>

1.2 If Grid Control was installed using an existing database, the startup script will not exist on the previous mentioned location. To construct the database startup script follow the below steps
1.2.1.- Create the directory scripts
$mkdir $ORACLE_HOME/install/unix/scripts
1.2.2.- Create the file seedstup
$touch $ORACLE_HOME/install/unix/scripts/seedstup
1.2.3.- Open the file seedstup and enter the following text. Replace the environment variables and the user with the appropriate values.

#!/bin/sh
#Script to start and stop the Listener and Database during shutdown and restart of the machine

function dbop
{
todo=$1
case $todo in

startup)
$ORACLE_HOME/bin/lsnrctl start
$ORACLE_HOME/bin/sqlplus '/as sysdba'<<- _START_DB
startup;
exit;
_START_DB
;;

shutdown)

$ORACLE_HOME/bin/lsnrctl stop;
$ORACLE_HOME/bin/sqlplus '/as sysdba'<<- _STOP_DB
shutdown immediate;
exit;
_STOP_DB
;;

*)
echo $"Usage: $0 {start|stop}";
exit 1;
esac

}

export ORACLE_HOME=<ORACLE_HOME_PATH>
user=<OS_user_that_owns_the_installation>
export ORACLE_SID=<SID>
SU=/bin/su

if [ $# -gt 1 ]; then
dbop $2
export -n ORACLE_HOME
export -n ORACLE_SID
exit
fi

case "$1" in
start)
$SU $user -c "$ORACLE_HOME/install/unix/scripts/seedstup $1 startup"
;;
stop)
$SU $user -c "$ORACLE_HOME/install/unix/scripts/seedstup $1 shutdown"
;;
*)
echo $"Usage: $0 {start|stop}";
exit 1;
esac

export -n ORACLE_HOME
export -n ORACLE_SID

$chmod +x seedstup
B.- Checking the oms and agent startup scripts
B.1 Open the file
$OMS_ORACLE_HOME/install/unix/scripts/omsstup
and check that the ORACLE_HOME and user are correctly set
B.2 Open the file
$AGENT_HOME/install/unix/scripts/agentstup
and check that the ORACLE_HOME and user are correctly set

C.- Configure the gcstartup script
Grid Control installation installs the gcstartup script and the appropriate links to it to process it on the preferred runlevel. The locations are
/etc/rc.d/init.d/gcstartup
/etc/rc.d/rc5.d/S99gcstartup
/etc//rc.d/rc3.d/S99gcstartup

The two files on the bottom should be hard links to the first script.
On each new installation or modification of the Grid Control components, lines are added to gcstartup which may cause malfunctioning of the script. For the script to work properly, edit the script following the steps below.

C.1. Please backup the three files mentioned above. Then edit the gcstartup script to contain only the
following text:

Replace <AGENT_HOME>, <DB_HOME> and <OMS_HOME> with the complete path to each home.

D.- Testing the script
Shutdown all Grid Control components
execute the command
/etc/rc.d/init.d/gcstartup start

All components should start in the correct order

II. Automatically Stopping Grid Control components

To stop the Grid Control Components automatically upon host shutdown or reboot, please follow the
next steps. It is assumed that you have already applied the above steps to ensure automatic
startup of the components.

1.- Open the gcstartup script, located on
/etc/rc.d/init.d/gcstartup

2.- Edit the script replacing its contents by the following text.

#!/bin/sh
#Source Function Library

if [ -f /etc/init.d/functions ]
then
. /etc/init.d/functions
else
if [ -f /etc/rc.d/init.d/functions ]
then
. /etc/rc.d/init.d/functions
fi
fi

case "$1" in
start)

if [ -f <DB_HOME>/install/unix/scripts/seedstup ]; then
. <DB_HOME>/install/unix/scripts/seedstup
fi

if [ -f <OMS_HOME>/install/unix/scripts/omsstup ]; then
. <OMS_HOME>/install/unix/scripts/omsstup
fi

if [ -f <AGENT_HOME>/install/unix/scripts/agentstup ]; then
. <AGENT_HOME>/install/unix/scripts/agentstup
fi

touch /var/lock/subsys/gcstartup
;;
stop)

if [ -f <OMS_HOME>/install/unix/scripts/omsstup ]; then
. <OMS_HOME>/install/unix/scripts/omsstup
fi

if [ -f <AGENT_HOME>/install/unix/scripts/agentstup ]; then
. <AGENT_HOME>/install/unix/scripts/agentstup
fi

if [ -f <DB_HOME>/install/unix/scripts/seedstup ]; then
. <DB_HOME>/install/unix/scripts/seedstup
fi
rm -f /var/lock/subsys/gcstartup

esac

Replace <AGENT_HOME>, <DB_HOME> and <OMS_HOME> with the complete path to each home.

This modification ensures that the same gcstartup script can be used to start and stop the components in the right order. Furthermore, in order to shutdown services, Red Hat Linux requires that each service creates a file under the /var/log/subsys/ directory to notify the system that the service is active. The above script
creates that file on startup and removes it when stopping the components.
Only the first time you shutdown or reboot it will be necessary to
create that file manually, using the following command
$touch /var/lock/subsys/gcstartup

4.- Create the following symbolic links to include the gcstartup script on the shutdown and reboot
process

$ln -s /etc/rc.d/init.d/gcstartup /etc/rc.d/rc0.d/K01gcstartup
$ln -s /etc/rc.d/init.d/gcstartup /etc/rc.d/rc6.d/K01gcstartup

# ln -s /etc/init.d/gcstartup /etc/rc5.d/S98gcstartup
# ln -s /etc/init.d/gcstartup /etc/rc3.d/S98gcstartup

The K on the symbolic link indicates it is a Kill script and the 01 indicates its priority on the
shutdown process.

5.- Testing the script
Execute the commands
/etc/rc.d/init.d/gcstartup start
/etc/rc.d/init.d/gcstartup stop

The components should start and stop automatically on the correct order.

References

BUG:5383799 - OMS,AGENT AND REPOSITORY DATABASE DO NOT AUTO START ON REBOOT OF MACHINE
NOTE:392426.1 - Problem: gcstartup does not start Agent and Management Server during server reboot

Thursday, 10 December 2009

Windows Oracle Batch Files

Category : Home > Databases > Oracle

Scripts to backup Oracle database on Windows NT

Here's a set of scripts to do database export, full database backup (putting tablespaces in

backup mode, not using RMAN) and scan the backup logs on Windows NT. The datafiles are first

copied to another filesystem using ocopy and then backuped with ntbackup. You can also see

some examples of using DOS variables in sqlplus. These scripts are examples only, obviously

you would have to adjust them for your environment.

REM ==========================================================================

REM Script# 1

REM This is the main backup script to run with Windows Scheduler / at command

REM ==========================================================================

set dbconn=system/manager@SAMPLEDB

set admdir=c:\admin\daily\

set logdir=c:\admin\daily\out\

set oradir1=c:\oracle\oradata\SAMPLEDB\

set oradir2=d:\oracle\oradata\SAMPLEDB\

set oradir3=e:\oracle\oradata\SAMPLEDB\

set orabckdir=e:\backup\SAMPLEDB\

set orabckdir1=e:\backup\SAMPLEDB\c\

set orabckdir2=e:\backup\SAMPLEDB\d\

set orabckdir3=e:\backup\SAMPLEDB\e\

REM

REM save previous copies of backup logs

REM

del %logdir%backup2.txt

ren %logdir%backup1.txt backup2.txt

ren %logdir%backup.txt backup1.txt

echo "SAMPLEHOST overnight.bat backup commenced ..." >>%logdir%backup.txt

date /t >>%logdir%backup.txt

time /t >>%logdir%backup.txt

REM

REM Run full export of the database

REM

echo "Exporting SAMPLEDB" ... >> %logdir%backup.txt

d:\oracle\ora81\bin\exp.exe parfile=d:\oracle\admin\SAMPLEDB\exp\SAMPLEDBexp.par >>

%logdir%backup.txt

REM

REM Compress the export file with winzip command line utility

REM May also put a line to remove the old dump files here

REM

c:\progra~1\winzip\wzzip.exe e:\oracle\oradata\SAMPLEDB\export\SAMPLEDB_full.zip

e:\oracle\oradata\SAMPLEDB\export\SAMPLEDB_full.dmp

REM

REM Append the database export log to backup.txt

REM

type d:\oracle\admin\SAMPLEDB\exp\logs\SAMPLEDB_full.log >> %logdir%backup.txt

echo "Finished export of SAMPLEDB" >> %logdir%backup.txt

echo "Generating sql scripts for hot SAMPLEDB backup ..." >> %logdir%backup.txt

sqlplus -s %dbconn% @%admdir%sql\backup.sql >> %logdir%backup.txt

echo "Putting SAMPLEDB in backup mode ..." >> %logdir%backup.txt

REM

REM Put all tablespaces in backup mode

REM

sqlplus -s %dbconn% @%admdir%sql\BEGIN_SAMPLEDB_BACKUP.sql >> %logdir%backup.txt

REM

REM Save the database files' location in SAMPLEDB_files.txt

REM

dir %oradir1% > %orabckdir%SAMPLEDB_files.txt

dir %oradir2% >> %orabckdir%SAMPLEDB_files.txt

dir %oradir3% >> %orabckdir%SAMPLEDB_files.txt

REM

REM File listing from sqlplus

REM

sqlplus -s %dbconn% @%admdir%sql\dbfiles.sql >> %orabckdir%SAMPLEDB_files.txt

REM

REM Copy Oracle files to backup directory on disk

REM

D:\oracle\ora81\bin\ocopy %oradir1%* %orabckdir1% >> %logdir%backup.txt

D:\oracle\ora81\bin\ocopy %oradir2%* %orabckdir2% >> %logdir%backup.txt

D:\oracle\ora81\bin\ocopy %oradir3%* %orabckdir3% >> %logdir%backup.txt

REM

REM backup filesystems

REM

REM D: drive is backed up last because it has the archived redo logs

REM

ntbackup backup c:\ e:\ d:\ /v /e /b /hc:on /d "SAMPLEHOST Overnight backup" /t normal /l

"%logdir%backup.txt"

ntbackup eject

date /T >> %logdir%backup.txt

time /T >> %logdir%backup.txt

echo "Putting SAMPLEDB in normal mode" >> %logdir%backup.txt

sqlplus -s %dbconn% @%admdir%sql\END_SAMPLEDB_BACKUP.sql >> %logdir%backup.txt

echo "SAMPLEHOST backup finished ..." >>%logdir%backup.txt

date /t >>%logdir%backup.txt

time /t >>%logdir%backup.txt

REM

REM Scan the logs for errors and produce the summary log file

REM

%admdir%SearchForErr.Bat %logdir%backup.txt

REM ==========================================================================

REM Script# 2

REM Script to sumbit the backup job using at command

REM ==========================================================================

@echo off

at \\SAMPLEHOST 23:30 /every:M,T,W,Th,F c:\admin\daily\overnight.bat

REM ==========================================================================

REM Script# 3

REM clean_archive.bat

REM Remove Oracle archived logs older than seven days

REM

REM NOTE: you need nt resource kit for this script to work

REM

REM ==========================================================================

C:\ntreskit\forfiles -pE:\oracle\oradata\SAMPLEDB\archive -d-7 -c"CMD /C del @FILE"

REM ==========================================================================

REM Script# 4

REM zip_archive.bat

REM Compress archived logs older than 2 days

REM

REM ==========================================================================

C:\ntreskit\forfiles -pD:\oracle\oradata\SAMPLEDB\archive -d-2 -c"CMD /C

c:\progra~1\winzip\wzzip.exe -amex -k @FILE.zip @FILE"

REM ==========================================================================

REM Script# 5

REM

REM SearchForErr.Bat

REM

REM This job will search the supplied file for error messages produce

REM a summary file if any are found.

REM

REM %1 - Supplied Output File to be checked.

REM c:\admin\daily\search.txt - Strings of text to be searched for.

REM c:\admin\daily\out\Error.txt - Errorlog of this job.

REM

REM ==========================================================================

@echo off

IF EXIST DEL c:\admin\daily\out\Summary.txt

echo . > c:\admin\daily\out\Summary.txt

echo Checking file - %1 >> c:\admin\daily\out\Summary.txt

echo . >> c:\admin\daily\out\Summary.txt

Findstr /i /g:\admin\daily\search.txt %1 >> c:\admin\daily\out\Summary.txt

REM ==========================================================================

REM

REM search.txt

REM File containing possible error messages to search for in the backup logs

REM Do not put this comments in search.txt file

REM

REM ==========================================================================

/* Server : SG_Q1 Date :

backup commenced

backup finished

Backup of

Backup started

Backup completed

\<* error *

^error *

\<* abort *

^abort *

\<* failed *

^failed *

\<* command *

^command *

Process Aborted

Attempt to fetch logical page

not to object

is already hashed.

was expected to be the first page of

Table Corrupt: Extent

Data size mismatch occurred

bytes found,

expected data length

Could not find row in Sysindexes

CHECKTABLE

Descriptor for system table

hash table.

via RID failed

the last RID

Could not retrieve row

less than or equal

is not correctly

Sort failed because dpages

an incorrect value.

to correct the value

your command.

Table Corrupt: Offset table is incorrect

correct offset

is=

Table Corrupt: The row number and offset

check this page

number table=

Table Corrupt: Object id wrong;

in ext=

(name =

Allocation Discrepancy: Page

pg#=

on extent=

that has an illegal

The non_clustered leaf row

indexid

tried to remove resource lock

Unable to find buffer holding

dbid =

Table corrupt:

Table Corrupt: object id does not match

Table Corrupt: Extent id

but used bit off

is empty but

Status =

is not marked suspect.

I/O error detected

is incorrectly trying

lockid=

not found;

Table Corrupt: The index id

Table Corrupt: Keys

check row

page number

Table Corrupt: Page is linked but not

extent id=

Extent not within segment

is not in segment

uninitialized pages encountered

Could not find virtual page

System error detected

error log

Memmove() was called with

Table Corrupt: A page is linked

page:page number=

Table Corrupt: Page linkage

pointing to this page=

this page=

Table Corrupt: Object id

Database Corrupt: The last checkpoint

different checkpoint

Table Corrupt: Extent structures

extent#=

be=

cannot be opened

The SA can drop

Cannot deallocate extent

does not match

deallocated.

Could not find leaf row

data row from logical data page

page split.

There is insufficient system memory

Table Corrupt: Type id

Could not find row in Sysdatabases

CHECKTABLE

Your server command (process

deadlocked

\<command\>

not match

following page:

should be

Table Corrupt: The values in adjust table

starting from the end of the table

table in this row

Verify started

Verify completed

ORA-

-- ==========================================================================

-- dbfiles.sql

-- SQL script to get list of all database files, redo log members

-- and control files

-- ==========================================================================

SELECT 'Data files' FROM DUAL;

SELECT name FROM v$datafile;

SELECT 'Redo logs' FROM DUAL;

SELECT member FROM v$logfile;

SELECT 'Control files' FROM DUAL;

SELECT name FROM v$controlfile;

EXIT;

-- ==========================================================================

-- backup.sql

-- SQL script to generate scripts to start and end database backup

-- ==========================================================================

set pagesize 0 echo off feedback off verify off

spool C:\admin\daily\sql\BEGIN_SAMPLEDB_BACKUP.sql

SELECT 'ALTER SYSTEM SWITCH LOGFILE;'

FROM dual;

SELECT 'ALTER TABLESPACE ' || tablespace_name || ' BEGIN BACKUP;'

FROM dba_tablespaces;

SELECT 'ALTER DATABASE BACKUP CONTROLFILE TO TRACE;'

FROM dual;

SELECT 'EXIT;'

FROM dual;

spool off

spool C:\admin\daily\sql\END_SAMPLEDB_BACKUP.sql

SELECT 'ALTER TABLESPACE ' || tablespace_name || ' END BACKUP;'

FROM dba_tablespaces;

SELECT 'ALTER SYSTEM SWITCH LOGFILE;'

FROM dual;

SELECT 'EXIT;'

FROM dual;

spool off

EXIT;