Oracle Data Guard

Customer Documentation
 Data Guard Customer Documentation

Copyright

 2013 MICROS Systems, Inc. All rights reserved. No part of this publication may be reproduced, photocopied, stored on a
retrieval system, or transmitted without the express prior written consent of the publisher. MICROS Systems, Inc. retains the
right to update or change the contents of this document without prior notice. MICROS Systems, Inc. assumes no responsibility
for the contents of this document.

OPERA is a trademark of MICROS Systems, Inc.

On Oracle and the On Oracle logo are trademarks of Oracle Corporation.

Information in this document is subject to change without notice.

MICROS Systems, Inc. makes no warranty of any kind with regard to this material, including but not limited to the implied
warranties of marketability and fitness for a particular purpose.

MICROS Systems, Inc. shall not be liable for errors contained herein or for incidental or consequential damages in connection
with the furnishing, performance, or use of this material.

Document Author: OPERA Global Technical Services

MICROS Systems, Inc.

7031 Columbia Gateway Drive
Columbia, MD 21046
(443) 285-8000 / Fax: (443)-285-6505

PAGE 2 of 39 MAY 14, 2013

 Data Guard Customer Documentation

Table of Contents

Introduction – Oracle Data Guard .................................................................................................................................6

Data Guard configurations ........................................................................................................................................6
Primary Database ..................................................................................................................................................6
Physical Standby Database ....................................................................................................................................6
Data Guard protection mode ....................................................................................................................................6
D:\scripts .......................................................................................................................................................................7
Opera Batch Runner Service ..........................................................................................................................................7
OPERA Batch Runner Schedule .................................................................................................................................8
Local Administrator user for OPERA Batch Runner Service ......................................................................................9
Standby Database – Start / Stop Standby ....................................................................................................................10
Start Standby Database ...........................................................................................................................................10
Stop Standby Database ...........................................................................................................................................11
Monitoring Standby Database – Daily Report .............................................................................................................11
Daily Report .............................................................................................................................................................11
Opera Data Guard Daily Report ...................................................................................................................................11
Opera Data Guard Daily Report...........................................................................................................................11
Standby Database ..............................................................................................................................................11
General Database Information: ...................................................................................................................11
Applied logs: ....................................................................................................................................................12
Archive gaps: ...................................................................................................................................................12
Logs not applied: ............................................................................................................................................12
Deleted archive logs: .....................................................................................................................................12
Process on standby server: .........................................................................................................................12
Production Database .........................................................................................................................................12
General Database Information: ...................................................................................................................12
Opera Version Information: ..........................................................................................................................13
Oracle version Information: .........................................................................................................................13
Tablespace usage: .........................................................................................................................................13
Number of database connections: .............................................................................................................14
List of last 3 days backups: .........................................................................................................................14
FAILED Daily Report – Standby down ..........................................................................................................................14
Opera Data Guard Daily Report...........................................................................................................................14
Standby Database ..............................................................................................................................................15
General Database Information: ...................................................................................................................15
Applied logs: ....................................................................................................................................................15

PAGE 3 of 39 MAY 14, 2013

 Data Guard Customer Documentation

Archive gaps: ...................................................................................................................................................15

Logs not applied: ............................................................................................................................................15
Deleted archive logs: .....................................................................................................................................15
Process on standby server: .........................................................................................................................15
FAILED Daily Report - archives not applied .................................................................................................................16
Opera Data Guard Daily Report...........................................................................................................................16
Standby Database ..............................................................................................................................................16
General Database Information: ...................................................................................................................16
Applied logs: ....................................................................................................................................................16
Archive gaps: ...................................................................................................................................................16
Logs not applied: ............................................................................................................................................16
Deleted archive logs: .....................................................................................................................................16
Process on standby server: .........................................................................................................................17
FAILED Daily Report – archive gap ...............................................................................................................................17
Opera Data Guard Daily Report...........................................................................................................................17
Standby Database ..............................................................................................................................................17
General Database Information: ...................................................................................................................17
Applied logs: ....................................................................................................................................................17
Archive gaps: ...................................................................................................................................................17
Logs not applied: ............................................................................................................................................17
Deleted archive logs: .....................................................................................................................................18
Process on standby server: .........................................................................................................................18
FAILED Daily Report – Production Invalid objects .......................................................................................................18
Opera Data Guard Daily Report...........................................................................................................................18
Production Database .........................................................................................................................................18
Daily Report – email distribution list ...........................................................................................................................18
Daily Report email address distribution list ............................................................................................................18
Monitoring Standby Database – Synchronize Standby................................................................................................19
Delete standby applied archives .............................................................................................................................19
Standby Application Server Synchronization ..........................................................................................................19
Data Guard Validation .................................................................................................................................................20
Standby Server troubleshooting .............................................................................................................................20
Standby Server monitoring .....................................................................................................................................21
Primary Server .........................................................................................................................................................22
Version Upgrades ........................................................................................................................................................22
Stop Standby Database ...........................................................................................................................................22
Stop OPERA Batch Runner Service ..........................................................................................................................23

PAGE 4 of 39 MAY 14, 2013

 Data Guard Customer Documentation

OPERA upgrades ......................................................................................................................................................23

Oracle upgrades ......................................................................................................................................................23
Manual Pre-Switchover Steps......................................................................................................................................23
OXI Servers ..............................................................................................................................................................23
IFC Servers ...............................................................................................................................................................24
Data Guard Switchover ................................................................................................................................................24
Primary Database ....................................................................................................................................................24
Standby Database ...................................................................................................................................................28
Data guard Failover .....................................................................................................................................................30
Failover ....................................................................................................................................................................30
Standby Database ...................................................................................................................................................30
Post-Switchover and Post-Failover Steps ....................................................................................................................33
Backups ...................................................................................................................................................................33
OXI Servers ..............................................................................................................................................................33
IFC Servers ...............................................................................................................................................................33
SYS and SYSTEM Password Change .............................................................................................................................34
Changing SYS and SYSTEM password ......................................................................................................................34
Antivirus .......................................................................................................................................................................35
Antivirus EXCLUSIONS .............................................................................................................................................35
Creating / Rebuilding the Standby Database Server Scripts ........................................................................................36
rebuild_DG.cmd ......................................................................................................................................................36
Log files ........................................................................................................................................................................37
Files to gather for support ...........................................................................................................................................38
Contact Micros Support ..........................................................................................................................................38
alertOPERA.log ........................................................................................................................................................38
System Files .............................................................................................................................................................39
Data Guard report ...................................................................................................................................................39
Daily Report .............................................................................................................................................................39

PAGE 5 of 39 MAY 14, 2013

 Data Guard Customer Documentation

Introduction – Oracle Data Guard

Oracle Data Guard ensures high availability, data protection, and disaster recovery for enterprise data. The basic
idea behind Data guard is that each Oracle database functions either in a primary database role or in a standby
database role - with the ability to transition from one role to the other. Data Guard maintains these standby
databases as transactional consistent copies of the production database. In case of a loss of the Primary Database,
the Standby can take over the Primary Role and act as a Primary Database.

OPERA utilizes a physical standby database that replicates the exact contents of its primary database across the
Oracle Net network layer. While the relative physical storage locations can differ, the data in the database will be
exactly the same as the primary database. It can function either in managed-recovery mode or in read-only mode,
but not in both modes at the same time.

The log-writer network-server and archiver processes running on the primary database select archived redo logs
and send them to the standby-database host, where the remote file server background process within the Oracle
instance performs the task of receiving archived redo-logs originating from the primary database. In OPERA testing
it has been determined a minimum connection pipe of 40mb would be required bandwidth for shipping these logs
from production to the standby server. Oracle uses port 1521 by default for this communication.

The Difference between a Switchover and a Failover is that after a Failover, the Standby Database becoming a
Primary now, cannot switchback to become a Standby Database again. In Opposition a Switchover exchanges the
Database Roles (The Primary becomes a Standby and the Standby becomes a Primary). Switchovers can be
performed arbitrarily, a Failover only once.

OPERA is a physical data guard implementation using the maximum performance setting having the data file
names and directories the same on the standby database as they are on the primary.

Data Guard configurations

A Data Guard configuration consists of one production database and up to nine standby databases. The databases
in a Data Guard configuration are connected by Oracle Net and may be dispersed geographically.

Primary Database
A Data Guard configuration contains one production database, also referred to as the primary database, which
functions in the primary role. This is the database that is accessed by most of your applications.

Physical Standby Database

OPERA uses Oracle Data Guard in a Physical Standby configuration. A physical standby database replicates the
exact contents of its primary database across the Oracle Net network layer, so the data in the database will be
exactly the same as the primary database. It can function either in managed recovery mode or in read-only mode,
but not in both modes at the same time. A standby database is initially created from a backup copy of the primary
database. Once created, Data Guard automatically maintains the standby database by transmitting primary
database redo data to the standby system and then applying the redo logs to the standby database.

Data Guard protection mode

Maximum Performance
This protection mode provides the highest level of data protection that is possible without affecting the
performance of a primary database. This is accomplished by allowing transactions to commit as soon as all redo
data generated by those transactions has been written to the online log. Redo data is also written to one or more

PAGE 6 of 39 MAY 14, 2013

 Data Guard Customer Documentation

standby databases, but this is done asynchronously with respect to transaction commitment, so primary database
performance is unaffected by delays in writing redo data to the standby database(s).

This protection mode offers slightly less data protection than maximum availability mode and has minimal impact
on primary database performance.

This is the default protection mode for OPERA installations.

Opera Single Server Opera Single Server

Primary Standby
UID HP
UID HP
1 2 3 4 ProLiant 1 2 3 4 ProLiant
OVER
DL385 G7 OVER
DL385 G7
1 2 OVER
TEMP
TEMP
1 5 1 2 OVER
TEMP
TEMP
1 5
POWER
POWER POWER
POWER POWER
POWER POWER
POWER
SUPPLY
SUPPLY SUPPLY
SUPPLY POWER
POWER SUPPLY
SUPPLY SUPPLY
SUPPLY POWER
POWER
CAP
CAP CAP
CAP
DIMMS DIMMS
1 4 7 10 10 7 4 1 1 4 7 10 10 7 4 1
9 7 5 3 1 1 3 5 7 9 9 7 5 3 1 1 3 5 7 9
11 8 5 2 11 8 5 2

2 6 2 6
2 5 8 11 2 5 8 11

83 6 6 4 9 2 12 12 2 9 4 66 38 83 6 6 4 9 2 12 12 2 9 4 66 38

22 ONLINE 1 22 ONLINE 1
AMP
SPARE 1 AMP
SPARE 1
STATUS STATUS
PROC PROC PROC PROC
PROC MIRROR PROC PROC MIRROR PROC
FANS FANS
FANS 3 7 FANS 3 7
6 5 4 3 2 1 6 5 4 3 2 1
6 5 4 3 2 1 6 5 4 3 2 1

4 8 4 8

PRIMARY SERVER STANDBY SERVER

Roles: Roles:
Primary Application Server Standby Application Server
Primary Database Server Standby Database Server
(Oracle Data Guard)

USERS

D:\scripts
Scripts are located in d:\scripts of the primary and standby servers.

Opera Batch Runner Service

Opera batch runner is a service based utility that allows batch files to be run according to a schedule. This is a
windows service created on the STANDBY SERVER and will need to be run as a local administrator user.

The schedule is controlled via a jobs.txt file in the D:\scripts\tools\OperaBatchRunner directory. Notices from the
service will be written to the Windows application event log.

PAGE 7 of 39 MAY 14, 2013

 Data Guard Customer Documentation

The directory “d:\scripts” is on both the primary and standby servers. This folder will include the below subfolders

o Standby Failover
o Standby Switchover\Primary
o Standby Switchover\standby
o blat
o logs
o diag

The following scripts are scheduled in the Opera Batch Job Service on the standby server.

 d:\scripts\start_standby.bat - scheduled to execute “When my Computer starts”

This script will start the standby database in standby recovery mode in the event of a server reboot

 d:\scripts\daily_report.bat - scheduled on a daily basis to generate. This is a daily report that needs to be
monitored to verify the standby server is current with applying all logs of production

 d:\scripts\sync.bat - scheduled to run every 6 hours

This job is used to synchronize Opera application files (exports,imports, attachments and email
attachments), it will make a backup of runtimes and customizable_reports as well.

The following are the logs generated by the Opera Batch Job Service on the standby server.

 d:\scripts\logs\daily_report.html is where the daily report output is located.

 d:\scripts\logs\del_app_arch.log run by the daily_report.bat to delete archive logs that have been
applied to the standby database.

 d:\scripts\logs\sync.log run by the sync.bat script shows the success of the synchronization operation for
the application server files.

OPERA Batch Runner Schedule

To modify the scheduled interval of the tasks that run, modify D:\scripts\tools\OperaBatchRunner\jobs.txt file.

Review the following link for information on syntax for modifying the job schedule:
http://quartz-scheduler.org/documentation/quartz-1.x/tutorials/crontrigger

PAGE 8 of 39 MAY 14, 2013

 Data Guard Customer Documentation

Local Administrator user for OPERA Batch Runner Service

You must add a logon account on “Opera Batch Runner” service and for it to run and it must be run as a local or
domain user account created.

1. local user
- user must exist on both server with same username and password
- user must be added into ora_dba and Backup Operators group on both server

2. domain user
- user must be added into ora_dba and Backup Operators group on both server

PAGE 9 of 39 MAY 14, 2013

 Data Guard Customer Documentation

The service must be restarted for the changes to take effect.

Be sure to verify the service is started on the standby server. Verify the service is logging in with a valid
username/password.

Standby Database – Start / Stop Standby

Start Standby Database

If the database is not running start the standby by running double click on the desktop shortcut:

Enter the SYS password

PAGE 10 of 39 MAY 14, 2013

 Data Guard Customer Documentation

Stop Standby Database

If you want to stop the standby database cleanly for maintenance you can double click on the desktop shortcut:

Enter the SYS password

Note: if you stop the standby database manually please make sure to start it up after the maintenance period or
reboot the server.

Monitoring Standby Database – Daily Report

Daily Report

D:\scripts\daily_report.bat is a Daily DG report is set up to run by the OPERA batch runner service to generate
reports on standby server daily that provides information on the status of the primary and standby environments.
This report provides information if the standby server is up and running, if the standby is current with production,
and it deletes archive files that have been applied to the standby server. This report also provides information on
the primary production database, version information, tablespace usage, invalid objects, number of user
connetions and backup information.

To manually generate a report run the d:\scripts\daily_report.bat script.

The output is written to d:\scripts\logs\daily_report.html and will be emailed to the configured email address
daily. Review this file daily to validate data guard consistency with production.

****REVIEW THIS FILE DAILY****

Opera Data Guard Daily Report

Opera Data Guard Daily Report

Standby Database

General Database Information:

STAR SYSTE
INST_I DATABAS INSTANC DATABAS PROTECTION
STATUS HOST NAME T M
D E NAME E NAME E ROLE MODE
TIME DATE

PAGE 11 of 39 MAY 14, 2013

 Data Guard Customer Documentation

27- 29-
MAXIMUM
MOUNTE BJXCYOPRDG PHYSICAL APR- APR-
1 OPERA OPERA PERFORMANC
D 1 STANDBY 2013 2013
E
18:31 06:00

Applied logs:

LOGS TIME
Last Applied : 29-APR-2013 03:07
Last Received : 29-APR-2013 03:07

Archive gaps:

GAPS
0

Logs not applied:

NOT APPLIED
0

Deleted archive logs:

NAME SEQUENCE#
D:\ORACLE\ADMIN\OPERA\ARCHIVE\OPERAT1S0737585868234.ARC 234
D:\ORACLE\ADMIN\OPERA\ARCHIVE\OPERAT1S0737585868233.ARC 233
D:\ORACLE\ADMIN\OPERA\ARCHIVE\OPERAT1S0737585868230.ARC 230
D:\ORACLE\ADMIN\OPERA\ARCHIVE\OPERAT1S0737585868231.ARC 231
D:\ORACLE\ADMIN\OPERA\ARCHIVE\OPERAT1S0737585868232.ARC 232
D:\ORACLE\ADMIN\OPERA\ARCHIVE\OPERAT1S0737585868235.ARC 235
D:\ORACLE\ADMIN\OPERA\ARCHIVE\OPERAT1S0737585868237.ARC 237
D:\ORACLE\ADMIN\OPERA\ARCHIVE\OPERAT1S0737585868236.ARC 236

Process on standby server:

PROCESS STATUS
ARCH CLOSING
ARCH CLOSING
MRP0 APPLYING_LOG
RFS IDLE
RFS IDLE

Production Database

General Database Information:

PAGE 12 of 39 MAY 14, 2013

 Data Guard Customer Documentation

DATABASE DATABASE START SYSTEM

INST_ID INSTANCE STATUS HOST NAME
NAME ROLE TIME DATE
27-
29-APR-
APR-
1 OPERA OPERA OPEN BJXCYOPRDB1 PRIMARY 2013
2013
06:00
18:09

Opera Version Information:

OPERA VERSION
5.0.03.01 E00032

Oracle version Information:

VERSION
10.2.0.4 Patch 45
PLATFORM_NAME
Microsoft Windows x86 64-bit

Tablespace usage:

TABLESPACE SIZE (M) MAX SIZE (M) USED % TYPE STATUS

FINDATA 8,192.000 4,096.000 0.96 PERMANENT OK
FININDX 8,192.000 4,096.000 8.38 PERMANENT OK
LOGDATA 1,024.000 4,096.000 6.14 PERMANENT OK
LOGINDX 1,017.000 4,096.000 13.79 PERMANENT OK
NAMEDATA 1,024.000 4,096.000 1.07 PERMANENT OK
NAMEINDX 1,024.000 4,096.000 2.84 PERMANENT OK
OPERA_DATA 8,192.000 4,096.000 19.45 PERMANENT OK
OPERA_INDX 1,617.000 4,096.000 16.11 PERMANENT OK
OXI_DATA 768.000 4,096.000 4.39 PERMANENT OK
OXI_INDX 768.000 4,096.000 0.86 PERMANENT OK
QUICKDATA 517.000 4,096.000 0.46 PERMANENT OK
QUICKINDX 517.000 4,096.000 0.58 PERMANENT OK
RATEDATA 517.000 4,096.000 0.75 PERMANENT OK
RATEINDX 517.000 4,096.000 1.23 PERMANENT OK
RESVDATA 517.000 4,096.000 1.18 PERMANENT OK
RESVINDX 517.000 4,096.000 2.17 PERMANENT OK
SYSAUX 290.000 2,048.000 10.50 PERMANENT OK
SYSTEM 2,048.000 2,048.000 55.23 PERMANENT OK
TEMPSEG 2,048.000 1,900.000 TEMPORARY OK
TOOLS 151.250 2,048.000 0.01 PERMANENT OK

PAGE 13 of 39 MAY 14, 2013

 Data Guard Customer Documentation

UNDOTBS1 2,048.000 1,900.000 1.75 UNDO OK

------------- -------------
Total: 41,505.250 75,480.000

Number of database connections:

INSTANCE DATABASE CONNECTIONS

1 1

List of last 3 days backups:

SESSION OUTPUT TIME OUT

START_TIME END_TIME STATUS INPUT_TYPE DAY
RECID (M) TAKEN INST
26-APR-2013 26-APR-
1 408 COMPLETED DB FULL Friday 00:01:03
14:13 2013 14:14
27-APR-2013 27-APR-
18 0 FAILED ARCHIVELOG Saturday 00:00:04
14:02 2013 14:02
COMPLETED
27-APR-2013 27-APR-
20 3,312 WITH DB INCR Saturday 00:04:24
14:05 2013 14:09
WARNINGS
27-APR-2013 27-APR-
25 9 COMPLETED ARCHIVELOG Saturday 00:01:38
14:17 2013 14:19
27-APR-2013 27-APR-
28 305 COMPLETED DB FULL Saturday 00:00:56
15:42 2013 15:43
28-APR-2013 28-APR-
41 51 COMPLETED ARCHIVELOG Sunday 00:01:31 1
11:00 2013 11:01
28-APR-2013 28-APR-
44 15 COMPLETED ARCHIVELOG Sunday 00:01:31 1
15:00 2013 15:01
28-APR-2013 28-APR-
47 15 COMPLETED ARCHIVELOG Sunday 00:01:28 1
19:00 2013 19:01
29-APR-2013 29-APR-
50 20 COMPLETED ARCHIVELOG Monday 00:00:50 1
00:08 2013 00:08
COMPLETED
29-APR-2013 29-APR-
53 3,082 WITH DB INCR Monday 00:03:42 1
03:00 2013 03:04
WARNINGS
29-APR-2013 29-APR-
58 9 COMPLETED ARCHIVELOG Monday 00:01:10 1
03:06 2013 03:08

FAILED Daily Report – Standby down

Opera Data Guard Daily Report

PAGE 14 of 39 MAY 14, 2013

 Data Guard Customer Documentation

Standby Database

General Database Information:

select a.inst_id,a.name "DATABASE NAME", upper(c.instance_name) "INSTANCE
NAME", ''||c.status||'' "STATUS",
*
ERROR at line 1:
ORA-01034: ORACLE not available
declare
*
ERROR at line 1:
ORA-01034: ORACLE not available
Applied logs:
select 'Last Applied : ' Logs, to_char(next_time,'DD-MON-YYYY HH24:MI') Time
*
ERROR at line 1:
ORA-01034: ORACLE not available
Archive gaps:
select ''||
*
ERROR at line 1:
ORA-01034: ORACLE not available
declare
*
ERROR at line 1:
ORA-01034: ORACLE not available
Logs not applied:
select count(*) "NOT APPLIED"
*
ERROR at line 1:
ORA-01034: ORACLE not available
declare
*
ERROR at line 1:
ORA-01034: ORACLE not available
Deleted archive logs:
declare
*
ERROR at line 1:
ORA-01034: ORACLE not available
select name, sequence#
*
ERROR at line 1:
ORA-01034: ORACLE not available
Process on standby server:

PAGE 15 of 39 MAY 14, 2013

 Data Guard Customer Documentation

select process, status from v$managed_standby

*
ERROR at line 1:
ORA-01034: ORACLE not available

FAILED Daily Report - archives not applied

Opera Data Guard Daily Report

Standby Database

General Database Information:

STAR SYSTE
INST_I DATABAS INSTANC DATABAS PROTECTION
STATUS HOST NAME T M
D E NAME E NAME E ROLE MODE
TIME DATE
27- 27-
MAXIMUM
MOUNTE BJXCYOPRDG PHYSICAL APR- APR-
1 OPERA OPERA PERFORMANC
D 1 STANDBY 2013 2013
E
14:44 14:45

Applied logs:

LOGS TIME
Last Applied : 27-APR-2013 14:35
Last Received : 27-APR-2013 14:45

Archive gaps:

GAPS
0

Logs not applied:

NOT APPLIED
34
(Some archive logs are not applied, please
contact OPERA support.)

Deleted archive logs:

No log files were deleted.

PAGE 16 of 39 MAY 14, 2013

 Data Guard Customer Documentation

Process on standby server:

PROCESS STATUS
ARCH CLOSING
ARCH CLOSING
MRP0 WAIT_FOR_GAP
RFS IDLE
RFS IDLE

FAILED Daily Report – archive gap

Opera Data Guard Daily Report

Standby Database

General Database Information:

STAR SYSTE
INST_I DATABAS INSTANC DATABAS PROTECTION
STATUS HOST NAME T M
D E NAME E NAME E ROLE MODE
TIME DATE
27- 27-
MAXIMUM
MOUNTE BJXCYOPRDG PHYSICAL APR- APR-
1 OPERA OPERA PERFORMANC
D 1 STANDBY 2013 2013
E
15:24 15:27

Applied logs:

LOGS TIME
Last Applied : 27-APR-2013 15:04
Last Received : 27-APR-2013 15:27

Archive gaps:

GAPS
5
(Your databases are not in sync, please
contact OPERA support.)

Logs not applied:

NOT APPLIED

PAGE 17 of 39 MAY 14, 2013

 Data Guard Customer Documentation

5
(Some archive logs are not applied, please
contact OPERA support.)

Deleted archive logs:

No log files were deleted.

Process on standby server:

PROCESS STATUS
ARCH CLOSING
ARCH CLOSING
MRP0 WAIT_FOR_GAP
RFS IDLE
RFS IDLE

FAILED Daily Report – Production Invalid objects

Opera Data Guard Daily Report

Production Database

Daily Report – email distribution list

Daily Report email address distribution list

To modify/add/remove email addresses in the daily report distribution list open D:\scripts\setenv.bat and update
the SET TO-ADDRESS line, comma delimited for multiple addresses in “double quotes”.

PAGE 18 of 39 MAY 14, 2013

 Data Guard Customer Documentation

Update this on both primary and standby servers.

Monitoring Standby Database – Synchronize Standby

Delete standby applied archives

The daily report script also produces d:\scripts\del_app_arch.log that runs daily to delete archive logs that have
been applied to the standby database. Please review this report to verify archive logs are purging. You are looking
that the validation succeeded for all achieve logs and Recovery Manager complete.

You should also monitor the disk space location of the archive logs on the standby d:\oracle\admin\opera\archive.

Standby Application Server Synchronization

Review d:\scripts\sync.log for verification that the application server updates are being copied over to the standby
server.

This scheduled task will copy the following files from primary to the standby server:

 D:\MICROS\opera\export\
 D:\MICROS\opera\import\
 D:\oracle\oradata\attachments\
 D:\oracle\oradata\email_attachments\

PAGE 19 of 39 MAY 14, 2013

 Data Guard Customer Documentation

 D:\MICROS\opera\Production\Runtimes\ to D:\MICROS\opera\Production\Runtimes_prod\
 D:\MICROS\opera\Production\Customizable_Reports\ to
D:\MICROS\opera\Production\Customizable_Reports_prod\

Data Guard Validation

Standby Server troubleshooting

1. Restarting the standby database if any of the above monitoring has any gaps or errors.
First run the STOP_DG desktop shortcut
Secondly run START_DG desktop shortcut

2. Manually generate the daily_report run the d:\scripts\daily_report.bat script.

3. Review the output that is written to d:\scripts\logs\daily_report.html and verify there are no errors or
gaps.
4. Review the D:\oracle\admin\OPERA\bdump\alert_opera.log for any errors on applying archive logs to the
standby.

There are several things to review to validate data guard is current.

Login to the primary database as “SYS” user and issue the below commands:
alter system archive log current;
archive log list;

The above screenshot shows that the current online log sequence# is 41.

Login to the standby database as “SYS” user and issue the below SQL commands.
select process, status, sequence# from v$managed_standby;

The above screenshot confirms that the log sequence# 41 is currently being applied to the standby.
select thread#, sequence#, applied from v$archived_log order by sequence#;

PAGE 20 of 39 MAY 14, 2013

 Data Guard Customer Documentation

Standby Server monitoring

1. d:\scripts\logs\daily_report.html – Verify there are no archive gaps and all logs are applied
2. d:\scripts\del_app_arch.log – Verify the archives applied are purging on the standby
3. d:\scripts\sync.log – Verify the application files are current
4. Verify that the archived redo log was applied
SQL > select sequence#, applied from v$archived_log order by sequence#;

If the applied column has a ‘Y’, those are the archives applied

Alternately run

SQL > select sequence#, applied from v$archived_log where applied=’N’ order by sequence#;

to show logs that have not applied.

This query will show the time of the last log applied:

select 'Last applied : ' Logs, to_char(next_time,'DD-MON-YY:HH24:MI:SS') Time from v$archived_log

where sequence# = (select max(sequence#) from v$archived_log where applied='YES') union select 'Last
received : ' Logs, to_char(next_time,'DD-MON-YY:HH24:MI:SS') Time from v$archived_log where
sequence# = (select max(sequence#) from v$archived_log);

5. Verify the Database mode

SQL > Select controlfile_type, open_mode from v$database;

6. Verify that there are no gaps in the logs being applied

SQL > Select * from v$archive_gap;

PAGE 21 of 39 MAY 14, 2013

 Data Guard Customer Documentation

Primary Server

1. Verify the backups are running and archives are being groomed as expected
2. Verify production is started:
3. Verify standby is started:
SQL > Select controlfile_type, open_mode from v$database;

This command should state that your Controlfile_type is “OPEN” and the database should be “READ
WRITE”

Version Upgrades
NOTE – BE SURE TO NOTIFY OPERA SUPPORT PRIOR TO AN UPGRADE THAT THIS IS A DATA
GUARD ENVIRONMENT TO SHUTDOWN STANDBY PRIOR TO UPGRADES.

Prior to any OPERA or Oracle upgrade shutdown the standby database. This can be used as a pre-upgrade image in
the event of failure and the need to revert to the pre-upgrade standby version. This would require a failover to
take place so the upgrade is not replicated to the standby.

Stop Standby Database

If you want to stop the standby database cleanly for maintenance you can double click on the desktop shortcut:

Enter the SYS password

PAGE 22 of 39 MAY 14, 2013

 Data Guard Customer Documentation

Stop OPERA Batch Runner Service

Stop the Opera Batch Service

OPERA upgrades

OPERA upgrades are run on the primary/production database server. Once successful a runtimes only upgrade will
need to be run on the standby database server. Once the OPERA upgrade is completely successful, start the
standby server back up and the database updates will be replicated to the standby database automatically.

***CAUTION*** Monitor the archive log destination on both servers for sufficient free space during all OPERA
upgrades.

Oracle upgrades

The Oracle binary installation upgrade will need to be performed on both the primary and standby servers.

However the database upgrade is only run on the primary/production database. Once the upgrade is successful
start the standby server back up and the database updates will be replicated to the standby database
automatically.

Manual Pre-Switchover Steps

These steps apply anytime the Primary database is moved to the Standby server for a planned Standby
Switchover.

OXI Servers

Stop the OXI services:

PAGE 23 of 39 MAY 14, 2013

 Data Guard Customer Documentation

1. On each OXI server open Windows Services and stop all services that start with “Opera Interface for…”

IFC Servers

Stop the IFC services:

1. On each IFC server open Windows Services and stop the service “Opera IFC Controller”

Data Guard Switchover

A switchover is a role reversal between the primary database and standby database. Switchover operation
guarantees no data loss. This is typically done for planned maintenance of the primary system. During a
switchover, the primary database transitions to a standby database and the standby database transitions to the
primary. The transition occurs without having to recreate either database.

Primary Database

RUN SCRIPTS ON PRIMARY SERVER FIRST

On the production database server navigate to d:\scripts\Standby_Switchover\Primary and run as administrator

the Primary_to_standby.bat file.

Enter the SYS password:

PAGE 24 of 39 MAY 14, 2013

 Data Guard Customer Documentation

This script will perform the following actions:

1. Stops the application server and removes the application virtual IP from the server

2. Close the primary database

PAGE 25 of 39 MAY 14, 2013

 Data Guard Customer Documentation

3. Start it up as the standby database

PAGE 26 of 39 MAY 14, 2013

 Data Guard Customer Documentation

4. Starts the standby server OPERA batch runner service and starts the service

Note: app server service stopped as expected

5. Updates BGInfo on the server to denote the standby role

6. Adds the stop_dg and start_dg shortcuts
7. Review the log located at D:\scripts\standby_switchover\primary\primary_to_standby.log

You can also query the database and verify the mode is standby:
select controlfile_type, open_mode from v$database;
This command should state that your Controlfile_type is “STANDBY” and the database should be “MOUNTED”

PAGE 27 of 39 MAY 14, 2013

 Data Guard Customer Documentation

The following is the D:\scripts\standby_switchover\primary\primary_to_standby.bat log on the production

server:

 D:\scripts\standby_switchover\primary\primary_to_standby.log

RUN SCRIPTS ON PRIMARY FIRST

Standby Database

RUN SCRIPTS ON STANDBY SECOND AFTER PRIMARY SCRIPTS RUN SUCCESSFULLY

On the standby database server navigate to D:\scripts\Standby Switchover\standby as administrator run the
standby_to_primary.bat.

Enter the SYS password:

PAGE 28 of 39 MAY 14, 2013

 Data Guard Customer Documentation

This script will perform the following actions:

1. Stops and disables the OPERA Batch Runner service

2. Shuts down the standby database

3. Creates the application server virtual address on this node

4. Starts the application server on this node
5. Start up the database as the Primary database

6. Updates BGInfo on the server to denote the production role

7. Removes the stop_dg and start_dg shortcuts
8. Review the log located at D:\scripts\standby_switchover\standby\standby_to_primary.sql

PAGE 29 of 39 MAY 14, 2013

 Data Guard Customer Documentation

You can also query the database and verify the mode is OPEN:
select controlfile_type, open_mode from v$database;
This command should state that your Controlfile_type is “OPEN” and the database should be “READ WRITE”

At this point, the standby database can be used as the primary database by the application.

The following is the D:\scripts\standby_switchover\standby\standby_to_primary.bat log on the standby server:

 D:\scripts\standby_switchover\standby\standby_to_primary.log

Data guard Failover

CAUTION – CONTACT OPERA SUPPORT FOR DBA GUIDANCE

Failover

In case of a loss of the Primary Database, the Standby can take over the Primary Role and act as a Primary
Database. A Failover can be performed when all or most of the information until the unavailability of the Primary
Database was propagated to the Standby.

This should be run only from the standby server and only if the primary database is no longer available.

Once the old primary database server is available, it can be built as a standby database server. The configuration
can remain or if desired a scheduled switchover can be performed and the primary role can be put back on the
original primary server. Switching the roles back will require a short downtime.

Standby Database

Navigate to D:\scripts\Standby Failover from a command prompt window and execute the Failover.bat.

PAGE 30 of 39 MAY 14, 2013

 Data Guard Customer Documentation

Type Y and enter the sys password

This script performs the following actions:

1. Stops and disables the OPERA Batch Runner service

2. Shuts down the standby database
3. Creates the application server virtual address on this node
4. Start up the database as the Primary database

PAGE 31 of 39 MAY 14, 2013

 Data Guard Customer Documentation

5. Updates BGInfo on the server to denote the production role

6. Starts the application server on this node
7. Removes the stop_dg and start_dg shortcuts

8. Review the log located at D:\scripts\standby_failover\failover.log

You can also query the database and verify the mode is OPEN:
select controlfile_type, open_mode from v$database;
This command should state that your Controlfile_type is “OPEN” and the database should be “READ WRITE”.

PAGE 32 of 39 MAY 14, 2013

 Data Guard Customer Documentation

The following is the D:\scripts\standby_failover\failover.bat log on the standby server:

 D:\scripts\standby_failover\failover.log

Post-Switchover and Post-Failover Steps

These steps apply anytime the Primary database is moved to the Standby server, either via the Standby Switchover
or the Standby Failover procedure.

Backups

All backup jobs need to be configured to point to the newly appointed Primary server and database.

OXI Servers

1. Go to \oracle\product\10.2.0\client_1\network\admin\tnsnames.ora

2. Update the HOST with the new production server name ON each OXI server

OPERA.WORLD, OPERA =
(DESCRIPTION =
(ADDRESS_LIST =
(ADDRESS = (PROTOCOL = TCP)(HOST = <PRODUCTION_DB_SERVER_NAME>)(PORT = 1521))
)
(CONNECT_DATA =
(SERVER = DEDICATED)
(SERVICE_NAME = OPERA)
)
)

3. On each OXI server open Windows Services and start all services that start with “Opera Interface for…”

IFC Servers

1. Go into the registry and update HKEY\LocalMachine\SOFTWARE\Micros-Fidelio\Opera IFC Controller

2. Update the key OperaIfcWS to the new server name ->
http:// <PRODUCTION_DB_SERVER_NAME>/Operajserv/Ifc8ws/Ifc8ws

PAGE 33 of 39 MAY 14, 2013

 Data Guard Customer Documentation

3. On each IFC server open Windows Services and start the service “Opera IFC Controller”

SYS and SYSTEM Password Change

Changing SYS and SYSTEM password

The following details how passwords should be changed in a Dataguard environment when you are planning to
change the SYS or SYSTEM password in PRIMARY

Step 1: change the password in primary database

SQL> alter user sys identified by <new password>;
SQL> alter user system identified by <new password>;

Step 2: Copy the password file to the standby database

1. Rename password file on standby
D:\oracle\1020\database\PWDOpera.ORA to D:\oracle\1020\database\PWDOpera.ORA_old

PAGE 34 of 39 MAY 14, 2013

 Data Guard Customer Documentation

2. Copy password file from production to the standby database server

D:\oracle\1020\database\PWDOpera.ORA

Step 3: Update the passwords in any other locations that reference it. Possible location:
1. Backup Exec change agent password on DB server
Start > Programs > Symantec Backup Execc > Backup Exec Remote Agent Utility
2. Change Backup Exec Server SYS password
3. Custom RMAN scripts
4. Windows scheduled tasks

If the password is already changed, and/or you are getting ORA-16191 during log shipping then follow the below
action.

Action: Copy the password file from PRIMARY to STANDBY (OR) recreate the password file in standby with the
same password.

After the password has been modified on the standby server(s), ensure that archive logs are being applied and no
errors seen in the alert log.

select sequence#,applied from v$archived_log order by sequence#;

Antivirus
Antivirus EXCLUSIONS

Micros Systems, Inc. does not promote, certify or support any specific antivirus software installed on Opera
servers. However Micros Systems, Inc. does not prohibit anti-virus software installed on Opera servers.

If a customer decides to install the anti-virus software on this Opera server, the anti-virus should not lock or
attempt to lock any Opera or Oracle files.

Most anti-virus software has the ability to skip the scanning of certain files. It is required that the anti-virus
software be configured to exclude the scanning of the Oracle data files, Oracle Home folders, Micros folders in the
following locations:

C:\Program Files\Oracle

C:\ or D:\ Micros – usually just app server

C:\ or D:\ Oracle
G:\Oracle – if applicable

Make certain that anti-virus software does not have the email scanning option installed. If the email scanning
option is installed, either remove the option entirely (disabling it is not enough), or de-install and re-install the
anti-virus software without the email scanning option on any database server.

If the Anti-Virus software comes with a firewall, the firewall should be disabled.

It is the sole responsibility of the customer to support and configure any anti-virus software installed on Opera
servers. It is also the sole responsibility of the customer to ensure any anti-virus software does not cause any
incompatibility issues or performance problems to the Server, its operating system or the Opera Application. The
customer is also responsible for keeping any anti-virus software updated, to research and resolve any known issues
with any anti-virus software and implement desired bug fixes.

PAGE 35 of 39 MAY 14, 2013

 Data Guard Customer Documentation

If there are performance issues on the server and it is thought to be related to the anti-virus software due to the
configuration or the way that it scans the system, then Micros will ask that it be disabled until the problem is
resolved. In some circumstances Micros will request that the anti-virus software be completely removed from the
system while troubleshooting an issue. Once the issue is identified and correct changes are made the anti-virus
software can be re-enabled.

Creating / Rebuilding the Standby Database Server Scripts

CAUTION – CONTACT OPERA SUPPORT FOR DBA GUIDANCE

rebuild_DG.cmd

D:\scripts\rebuild_DG\rebuild_DG.cmd is run on the STANDBY SERVER and creates a production backup on the
production server as d:\stdby_backup. That backup is then copied to the standby servrer as d:\stdby_backup, and
is then restored to the standby server to get dataguard up and running.

1. Go into d:\scripts directory at a DOS prompt

2. Run D:\scripts\rebuild_DG\rebuild_DG.cmd file

3. Enter the SYS password

PAGE 36 of 39 MAY 14, 2013

 Data Guard Customer Documentation

4. On the production database server the backup will be written to d:\stdby_backup directory.

5. Once the backup is complete on production it will use robocopy to copy the backup to the standby
database server written to d:\stdby_backup

6. Once the copy is complete it will perform a restore of the standby database. There is no need to remove
the existing/bad standby database datafile the script will handle replacing them.

7. Log file can be found in d:\scripts\rebuild_DG\logs folder on the standby database server. Please review
rebuild.log for any obvious error.

8. Once the restore is complete it will start the standby database in standby recovery modes

Log files
The following are the logs generated by the Opera Batch Job Service on the standby server.

 d:\scripts\logs\daily_report.html is where the daily report output is located.

PAGE 37 of 39 MAY 14, 2013

 Data Guard Customer Documentation

 d:\scripts\logs\del_app_arch.log run by the daily_report.bat to delete archive logs that have been
applied to the standby database.

 d:\scripts\logs\sync.log run by the sync.bat script shows the success of the synchronization operation for
the application server files.

The following is the rebuild data guard log on on the standby server:

 d:\scripts\rebuild_DG\logs\rebuild.log

The following is the d:\scripts\OS_changes.exe log on the production and standby server:

 d:\scripts\logs\OS_Changes.log

The following is the D:\scripts\install_PROD.exe log on the production server:

 d:\scripts\logs\PROD_install.log

The following is the D:\scripts\install_DG.exe log on the standby server:

 d:\scripts\logs\DG_install.log

The following is the D:\scripts\standby_switchover\primary\primary_to_standby.bat log on the production

server:

 D:\scripts\standby_switchover\primary\primary_to_standby.log

The following is the D:\scripts\standby_switchover\standby\standby_to_primary.bat log on the standby server:

 D:\scripts\standby_switchover\standby\standby_to_primary.log

The following is the D:\scripts\standby_failover\failover.bat log on the standby server:

 D:\scripts\standby_failover\failover.log

The following is the OPERA Batch Runner install log on the standby server:

 D:\scripts\tools\OperaBatchRunner\OperaBatchRunner.InstallLog

Files to gather for support

Contact Micros Support

1. Contact Micros Support at 1-800-937-2211 for any OPERA related issues.

2. http://usc.micros.com/casemanagement will take you to Clarify Self Support to open a case. You will
need a login.

alertOPERA.log

PAGE 38 of 39 MAY 14, 2013

 Data Guard Customer Documentation

D:\oracle\admin\OPERA\bdump\alert_opera.log from both primary and standby database servers.

System Files

1. System event logs saved in text format from each node

2. Application event logs saved in text format from each node

Data Guard report

Standby database run D:\scripts\Physical_standby_diagnostics.txt from sqlplus

D:>sqlplus sys as sysdba
SQL>@d:\scripts\Physical_standby_diagnostics.txt

This will generate a file of the format dgdiag_phystby_&&dbname&×tamp&&suffix in the directory structure you
run it from (in the above example it will generate at the base d:\ drive), attach this file to the case.

Primary database run d:\scripts\Primary_diagnostics.txt

D:>sqlplus sys as sysdba

SQL>@d:\scripts\ Primary_diagnostics.txt

This will generate a file of the format dg_prim_diag_&&dbname&×tamp&&suffix in the directory structure you run
it from (in the above example it will generate at the base d:\ drive), attach this file to the case.

Daily Report

From the standby database server provide the d:\scripts\logs\daily_report.html to the case.

PAGE 39 of 39 MAY 14, 2013