Practice 19-a Preparing the Environment for Oracle 19c RAC P a g e |1

Practice 19-a

Preparing the Environment for Oracle 19c RAC

Practice Overview
In this practice you will create two virtual appliances that will be used to create an Oracle 19c RAC
database on Linux 7.

In high level, you will perform the following steps:

• Download Oracle Software installation files

• Install Oracle VirtualBox release 6.0.22 or later

• Create and configure an Oracle Linux 7.8 64-bit VirtualBox appliance (srv1)

• Build up and configure the second node (srv2) by cloning srv1

• Create and configure the shared disks

Practice Environment Architecture

The following diagram shows the architecture of the machines that you will create in this
practice:

Oracle RAC Administration, a course by Ahmed Baraka

 Practice 19-a Preparing the Environment for Oracle 19c RAC P a g e |2

Practice Environment Preparation Procedure

A. Downloading Oracle Software Installation files

1. From Oracle site, download the following products:

• Oracle Database 19c (19.3) for Linux x86-64.

• Oracle Grid Infrastructure 19c (19.3) for Linux x86-64

At the time of this writing, these products can be downloaded from the following link:
https://www.oracle.com/database/technologies/oracle19c-linux-downloads.html

Note: do not download the rpm file. Download the zip files.

B. Installing the required software

2. Download and install the following software products in your hosting PC:
• Oracle VirtualBox version 6 for Windows (6.0.22 or later)

C. Creating and Configuring an Oracle Linux 7.8 64-bit VirtualBox

Appliance

3. Create a Linux-based VirtualBox appliance with the specifications as shown in the table below. This
is an Oracle VirtualBox appliance which has a fresh installation of Oracle Linux 7.8 installed on it.

- You can download a pre-built one from here. Make sure you download the file in the section
titled as “Oracle Linux 7.8 64-bit (Fresh Installation)”

- Alternatively, you can create one from scratch. The procedure to create an VM machine with
Linux 7.x is explained in many articles in the Internet. Just Google it!

Item Value
Hostname srv1
Memory 8 GB
Note: This is the minimum recommended value from Oracle
OS Oracle Linux 7.8 64-bit
Storage 100 GB (dynamic)
Note: this is a local storage. You will create the shared storage later in the
practice.

Oracle RAC Administration, a course by Ahmed Baraka

 Practice 19-a Preparing the Environment for Oracle 19c RAC P a g e |3

4. Make sure the virtual appliance is opened by Oracle VirtualBox Manager and is turned off.

5. Go to the appliance settings and change its name to srv1 and enter a description about it in the
description field.

6. Click on "Shared Folders" link in the right-hand pane. Add shared folder by pressing "plus" icon.
Then select path to the location of the oracle software installation folder, and mark the checkbox
"Auto-mount". You can change the "Folder Name", if you want to.

This folder will be used to easily exchange files between the hosting PC and Linux in the VM
machine. In the rest of this tutorial document, this folder will be referred to as the staging
folder.

7. Add network adapters to srv1 by performing the following steps:

• Eventually, you will have three network adapters created in srv1:

o The first one will be used for public connection (Adapter type: Host-only)

o The second one will be used for private connection (Adapter type: Internal Network)

o Third one will be used for the Internet connection (Adapter type: Bridged Adapter)

Oracle RAC Administration, a course by Ahmed Baraka

 Practice 19-a Preparing the Environment for Oracle 19c RAC P a g e |4

a. In Oracle VirtualBox, go to the network settings of your virtual appliance. Change the value of
the "Attached To" field to "Host-only Adapter".

"Host-only" connection in VirtualBox allows connecting the VM machine to other VM machines

as well as to the hosting PC. This adapter represents the public network.

b. Click on the "Adapter 2" tab and enable it and set it attached to "Internal Network". Name
this network "privnet".
In VirtualBox, the “Internal Network” configuration allows the VM machine to connect only to
other VM machines. This adapter will be used for private connection in the RAC architecture.

Oracle RAC Administration, a course by Ahmed Baraka

 Practice 19-a Preparing the Environment for Oracle 19c RAC P a g e |5

c. Click on the "Adapter 3", enable it, and attach to "Bridged Adapter". This adapter will be
used for connecting to the internet.

8. Start srv1 and login to it as root

Note: If you are using the pre-built virtual machine from my web site, the readme file contains
the root password. Readme file is available in my website download page.

9. If you are using a pre-built copy of the virtual machine (like the one available in my web site),
make sure the Guest Additions version is upgraded to the version of the VirtualBox you are using.

The pre-built virtual machine that is available in my site was created using version 6.0.22. If you
are using a later version of Oracle VirtualBox, you should update its VirtualBox Guest Additions.
To Update the VirtualBox Guest Additions in the virtual machine, perform the following steps:

a. In the VirtualBox window, login as root and click on Devices menu | Insert Guest Additions
CD image.

b. When the following window pops up, click on Run button

Oracle RAC Administration, a course by Ahmed Baraka

 Practice 19-a Preparing the Environment for Oracle 19c RAC P a g e |6

c. Wait for the installation to finish.

d. Reboot the machine and login to it as root.

e. Right click on the VirtualBox Additions CD icon and select Eject option.

10. If you have a firewall software installed into your hosting PC, configure it to allow the traffic to go
to and come from Oracle VirtualBox appliance.

The NICs (network interface adapters) added to srv1 are not connected by default when you startup
the system. In the following steps, you will connect the added NICs.

11. Click on the network icon in the upper right corner of the screen and connect the added two
NICS (enp0s8 and enp0s9).

Oracle RAC Administration, a course by Ahmed Baraka

 Practice 19-a Preparing the Environment for Oracle 19c RAC P a g e |7

12. Perform the following steps to set the network adapter IP addresses:

a. In the VirtualBox appliance window, open the Network Connections window

Applications | System Tools | Settings | Network

b. You should see the three adapters as shown in the following screenshot:

c. Click on the settings icon of the first NIC (enp0s3). Click on IPv4 tab and fill it in as
follows, then click on Apply button:

Oracle RAC Administration, a course by Ahmed Baraka

 Practice 19-a Preparing the Environment for Oracle 19c RAC P a g e |8

d. To make the new IP address take effect, turn off the adapter then then turn it on again, as
shown in the following screenshot:

e. Click on the settings icon of the second NIC (enp0s8). Click on IPv4 tab and fill it in as
follows, then click on Apply button:

f. Turn off the second adapter then then turn it on again

g. Click on the settings icon of the third NIC (enp0s9). Make sure the checkbox “Connect
automatically” is marked. Click on IPv4 tab make sure the IPv4 Method is set to DHCP.

h. Close the Network window.

Oracle RAC Administration, a course by Ahmed Baraka

 Practice 19-a Preparing the Environment for Oracle 19c RAC P a g e |9

13. Open a terminal window then modify the hosts file as follow:
In real life scenario, the SCAN addresses should not be defined in the hosts file. They should be
defined on the DNS server to round-robin 3 addresses of the same subnet as the subnet of your
public network. The same condition applies for the public IP address, hostname, and the VIP
addresses. In this practice, the hosts file is used as a replacement to the DNS server.

vi /etc/hosts

127.0.0.1 localhost.localdomain localhost

# Public
192.168.56.71 srv1.localdomain srv1
192.168.56.72 srv2.localdomain srv2

# Private
192.168.10.1 srv1-priv.localdomain srv1-priv
192.168.10.2 srv2-priv.localdomain srv2-priv

# Virtual
192.168.56.81 srv1-vip.localdomain srv1-vip
192.168.56.82 srv2-vip.localdomain srv2-vip

# SCAN (in production this should be configured in DNS)

192.168.56.91 srv-scan.localdomain srv-scan
192.168.56.92 srv-scan.localdomain srv-scan
192.168.56.93 srv-scan.localdomain srv-scan

14. Reboot the virtual machine.

15. Login to srv1 as root, open a terminal window and, and to make sure that the VM machine is
connected to the Internet by pinging google.com
ping -c 3 google.com

16. In the hosting PC, open a command prompt window and ping the srv1 public IP address.
Note: VIP and SCAN addresses are not pingable at this stage. They will be activated after
installing Oracle Grid Infrastructure software.
ping 192.168.56.71

Oracle RAC Administration, a course by Ahmed Baraka

 Practice 19-a Preparing the Environment for Oracle 19c RAC P a g e | 10

17. Configure srv1 in PuTTY. I recommend setting the “keepalives” option to 9 seconds, as shown in
the following screenshot:

18. Open a Putty session to srv1 and login as root user.

19. In the following steps, you will create Oracle software owner users and groups.
a. Create the Oracle Grid Infrastructure home owner user (grid). Oracle database owner is
already there (oracle).
groupadd asmadmin
groupadd asmdba
groupadd oinstall

useradd -u 54323 -g oinstall -G asmadmin,asmdba grid

b. Reset the oracle and grid user passwords. The practice documents assume that the
passwords are set to "oracle".
passwd oracle
passwd grid

c. Add oracle user to the asmdba group

usermod -a -G asmdba oracle

usermod -g oinstall oracle

d. Add oracle and grid accounts to vboxsf group.

The vboxsf group was created by VirtualBox Guest Additions and it allows its members to
access the shared folders in the hosting PC.
usermod -a -G vboxsf oracle
usermod -a -G vboxsf grid

Oracle RAC Administration, a course by Ahmed Baraka

 Practice 19-a Preparing the Environment for Oracle 19c RAC P a g e | 11

20. Run the following command to install the libraries that are required for Oracle software
installation.
Note: The command sets the kernel parameter variables as well.
yum install oracle-database-preinstall-19c

21. Run the following command to make the changes made on the kernel parameters take effect.
sysctl -p

22. Install and configure the ASM required packages by performing the following steps as root:

a. Install Oracle ASMLib packages

yum install oracleasm-support
yum install kmod-oracleasm

b. Configure and load the ASM kernel module (respond to the questions as highlighted in red)
oracleasm configure -i

Configuring the Oracle ASM library driver.

This will configure the on-boot properties of the Oracle ASM library
driver. The following questions will determine whether the driver is
loaded on boot and what permissions it will have. The current values
will be shown in brackets ('[]'). Hitting <ENTER> without typing an
answer will keep that current value. Ctrl-C will abort.
Default user to own the driver interface []: grid
Default group to own the driver interface []: oinstall
Start Oracle ASM library driver on boot (y/n) [n]: y
Scan for Oracle ASM disks on boot (y/n) [y]: y
Writing Oracle ASM library driver configuration: done

c. Load the oracleasm kernel module:

/usr/sbin/oracleasm init

Creating /dev/oracleasm mount point: /dev/oracleasm

Loading module "oracleasm": oracleasm
Configuring "oracleasm" to use device physical block size
Mounting ASMlib driver filesystem: /dev/oracleasm

Oracle RAC Administration, a course by Ahmed Baraka

 Practice 19-a Preparing the Environment for Oracle 19c RAC P a g e | 12

23. Create the directory in which the Oracle software will be installed.
mkdir -p /u01/app/oracle/product
chown -R oracle:oinstall /u01
chmod -R 775 /u01

mkdir -p /u01/app/grid
mkdir -p /u01/app/19.0.0/grid
chown -R grid:oinstall /u01/app/grid
chown -R grid:oinstall /u01/app/19.0.0/grid
chmod -R 775 /u01

Oracle RAC Administration, a course by Ahmed Baraka

 Practice 19-a Preparing the Environment for Oracle 19c RAC P a g e | 13

D. Building up and Configuring srv2 by Cloning srv1

In the following steps, you will clone srv1 and make changes on the cloned machine to convert it to
srv2.

24. Shutdown srv1

25. Make a clone of the virtual appliance srv1. Give the new machine the name srv2. When you
clone the machine, make sure to initialize the network cards.

a. In Oracle VirtualBox Manager, Right-click on the srv1, select Clone command.

b. Fill-in the displayed window as follows.

c. Click on Clone button.

The directories that contain srv1 and srv2 both of them should be under the provided parent
directory.

26. Activate the network cards.

While cloning srv2, the MAC address of its network cards got re-initialized. As a result, they got
de-activated. To re-activate the cards, perform the following:
a. Login to srv2 as root

b. In the VirtualBox appliance window, open the Network Connections window

Applications | System Tools | Settings | Network

Oracle RAC Administration, a course by Ahmed Baraka

 Practice 19-a Preparing the Environment for Oracle 19c RAC P a g e | 14

c. The network adapters should look like the following. Only one adapter is enabled and the other
two are offline.

d. Open the settings of the first adapter. Check its MAC address. Make sure it is the same as the
MAC address that you took note of.
If it is not the same as the new MAC address, click on Identify tab and select the correct MAC
address from the dropdown list.

Oracle RAC Administration, a course by Ahmed Baraka

 Practice 19-a Preparing the Environment for Oracle 19c RAC P a g e | 15

e. Click on IPv4 tab and change the IP address as follows, then click on Apply button:

f. Turn the adapter off then turn it on again so that the changes take effect.

g. Click on the plus symbol beside the second adapter

Oracle RAC Administration, a course by Ahmed Baraka

 Practice 19-a Preparing the Environment for Oracle 19c RAC P a g e | 16

h. You should see a window like the following. Select the correct MAC address from the drop list
for this adapter.

i. Click on IPv4 tab then fill in the IP address as follows, then click on Add button:

j. Click on the plus symbol beside the third adapter:

Oracle RAC Administration, a course by Ahmed Baraka

 Practice 19-a Preparing the Environment for Oracle 19c RAC P a g e | 17

k. Select the correct MAC address from the drop list for this adapter, then click on Add button:

27. To list the network adapters installed in the system and their status, run the following command:
Note: virbr0 is a virtual network card used by xen virtualization. You can just ignore it.
nmcli device status

28. Run the following command to change the hostname of the appliance to srv2
hostnamectl set-hostname srv2.localdomain

# to verify:
hostnamectl status

29. Start srv1

30. Login to every machine as root and make sure that they can ping each other.
The ping command in Linux by default keeps displaying its reply output. To stop it, you can press
the [Ctl]+[c] key combination. If you are using the command in a terminal window in a
VirtualBox window, the right [Ctl]+[c] shortcut will be recognized by Oracle VirtualBox as a
shortcut to switch the Scale mode. If you want to stop the output of the ping command, use the
Left [Ctl]+[c] shortcut instead.
ping -c 3 srv1
ping -c 3 srv1.localdomain
ping -c 3 srv1-priv
ping -c 3 srv1-priv.localdomain

ping -c 3 srv2
ping -c 3 srv2.localdomain
ping -c 3 srv2-priv
ping -c 3 srv2-priv.localdomain

Oracle RAC Administration, a course by Ahmed Baraka

 Practice 19-a Preparing the Environment for Oracle 19c RAC P a g e | 18

31. Open Putty and save a connection configuration of srv2 in it as follows:

a. Save the configuration of srv2 in Putty. You can use the srv2 public IP address to save its
connection configuration in Putty.

b. Set the "Keepalives" value to 9 seconds.

c. Change the font text color of srv2 connection to the light green. This is to make it easy for
you to distinguish between windows connected to srv1 and srv2.

d. Save the configuration.

Oracle RAC Administration, a course by Ahmed Baraka

 Practice 19-a Preparing the Environment for Oracle 19c RAC P a g e | 19

E. Create and configure the Shared Disks

In this section of the practice, you will create three disks which will be shared by srv1 and srv2.
Those disks will later be configured to be used by the ASM.

32. Shut down both virtual machines srv1 and srv2.

33. In Oracle VirtualBox, select srv1 VM, then click on "Settings" | "Storage" link | "SATA
Controller" icon | "Add Hard Disk" button.

34. In the pop-up window, click on "Create new disk" button:

35. In the "Hard disk file type" window, select "VDI (VirtualBox Disk Image)" then click on Next:

Oracle RAC Administration, a course by Ahmed Baraka

 Practice 19-a Preparing the Environment for Oracle 19c RAC P a g e | 20

36. In the "Storage on physical hard disk" window, select "Fixed size" then click on Next:

37. In the "File location and size" window, click on the "select folder" icon (marked with step
number 1 on the screenshot below), select the parent folder of the current virtual machine.
Change the disk name to DISK1.vdi, set its size to 10 GB, then click on "Create" button.

In the screenshot example below, instead of saving the disk in "F:\OracleVB_VMs\Oracle

RAC\srv1", it is saved in the folder "F:\OracleVB_VMs\Oracle RAC".

38. Repeat the steps above to create an additional two disks named "DISK2" and "DISK3", set the
size of each to 15 GB, and save them in the same parent folder where you created DISK1.

39. Click on OK button and close the storage settings.

Oracle RAC Administration, a course by Ahmed Baraka

 Practice 19-a Preparing the Environment for Oracle 19c RAC P a g e | 21

40. You should end up with three shared disks, as follows:

Disk Size Used for which ASM Disk Group?

DISK1 10 GB OCR
DISK2 15 GB DATA
DISK3 15 GB FRA

41. Change the type of the shared disks to "Shareable". Click on File menu | Virtual Media
Manager (Ctrl+D) | select DISK1 | change its type to Shareable option | click on Apply button

Note: You must click on Apply button after changing the type of each disk. Otherwise, the change
will not take effect.

42. Change the type of the other two disks (DISK2 and DISK3) to "Shareable".

43. Close the Virtual Media Manager window.

44. Attach the shared disks to the other VM using (srv2). In Oracle VirtualBox, select srv2 machine,
click on "Storage" page | click on the SATA controller | click on "Add hardisk" icon | click on
"Choose existing disk" button | select the shared disk file

45. Start srv1

46. In a Putty command prompt, login as root to srv1

47. List the disks as seen by srv1. You should see the shareable disks that have been added to the
machine (sdb, sdc, and sdd).
ls /dev/sd*

Oracle RAC Administration, a course by Ahmed Baraka

 Practice 19-a Preparing the Environment for Oracle 19c RAC P a g e | 22

48. Use the fdisk utility to create partitions in the new disks /dev/sdb , /dev/sdc and /dev/sdd

The sequence of your answers should be "n", "p", "1", "Return", "Return" and "w".
Following is an example:
[root@srv1 ~]# fdisk /dev/sdb
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF
disklabel
Building a new DOS disklabel with disk identifier 0xe3033fca.
Changes will remain in memory only, until you decide to write them.
After that, of course, the previous content won't be recoverable.

Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)

WARNING: DOS-compatible mode is deprecated. It's strongly recommended to

switch off the mode (command 'c') and change display units to
sectors (command 'u').

Command (m for help): n

Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-1305, default 1): [ENTER] pressed
Using default value 1
Last cylinder, +cylinders or +size{K,M,G} (1-1305, default 1305): [ENTER]
pressed
Using default value 1305

Command (m for help): w

The partition table has been altered!

Calling ioctl() to re-read partition table.

Syncing disks.

49. List the partitions that have been created.

ls /dev/sd*

50. Create the ASM disks

oracleasm createdisk DISK1 /dev/sdb1
oracleasm createdisk DISK2 /dev/sdc1
oracleasm createdisk DISK3 /dev/sdd1

51. Run the oracleasm command scandisks to refresh the ASMLib disk configuration.

oracleasm scandisks

52. Verify that the disks are visible to ASM

oracleasm listdisks

Oracle RAC Administration, a course by Ahmed Baraka

 Practice 19-a Preparing the Environment for Oracle 19c RAC P a g e | 23

53. Start srv2 and verify that the same disks are visible to ASM.

oracleasm listdisks

Oracle RAC Administration, a course by Ahmed Baraka

 Practice 19-a Preparing the Environment for Oracle 19c RAC P a g e | 24

Summary
In this practice you have created two virtual machines and configured three shared storage disks
connected to each machine.
You now have an environment ready to install Oracle Grid Infrastructure 19c software, Oracle
Database 19c software, and create an Oracle RAC database.

Oracle RAC Administration, a course by Ahmed Baraka