一体机网络构架:
一体机配置:
配置:KVM
-
Pull the KVM tray out from the front of the rack, and open it using the handle.
-
Touch the touch pad.
-
Toggle between the host and KVM interface by pressing the Ctrl key on the left side twice, similar to a double-click on a mouse.
-
Select Target Devices from the Unit View of the user interface. The number of sessions shown should be 22 for Exadata Database Machine Full Rack, 11 for Exadata Database Machine Half Rack, and 5 for Exadata Database Machine Quarter Rack. The number of sessions should be 18 for Oracle Exadata Storage Expansion Full Rack, 9 for Oracle Exadata Storage Expansion Half Rack, and 4 for Oracle Exadata Storage Expansion Quarter Rack.
Note:
If all sessions are not shown, then select IQ Adaptors from the Ports heading. Click the table heading, Port, to sort the sessions by port number. Note any missing items. The sessions are numbered from the bottom of the rack to the top. -
Return to the Target Devices screen.
-
Select Local from User Accounts.
-
Click Admin under Users.
-
Set a password for the Admin account. Do not modify any other parameters.
-
Click Save.
-
Select Network from Appliance Settings. The Network Information screen appears.
-
Enter the values for Address, Subnet, Gateway, and the IP addresses of the DNS servers.
-
Click Save.
-
Connect the KVM LAN1 Ethernet port to the management network.
-
Verify the port has been configured correctly by checking the MAC address on the Network Information screen. The address should match the label next to the LAN1/LAN2 ports on the rear of the KVM switch.
-
Select Overview from Appliance.
-
Enter a name for the KVM switch.
-
Click Save.
-
Restart the KVM switch by selecting Reboot under Overview.
-
Examine the firmware version of the switch by selecting Versions from Appliance Settings. There are two version numbers shown, Application and Boot, as shown in the following:
Required version is: Application 1.2.10.15038 Boot 1.6.15020
配置 KVM Switch to Access a Server
The following procedure describes how to configure the KVM switch to access the servers:
Note:
The KVM switch is only available in Oracle Exadata Database Machine X2-2 racks and Oracle Exadata Storage Expansion Racks.-
Select Target Devices from Unit View.
-
Power on the server. The power button is on the front panel. If the button seems stuck, then use a small tool to loosen the button.
-
Click the system name in the Name column using the left mouse button.
-
Click Overview, and overwrite the name with the Oracle standard naming format of customer prefix, node type, and number. For example,
trnacel03
has the prefixtrna
, and storage cell 3 from the bottom of the rack, andtrnadb02
has the prefixtrna
, and database server 2 from the bottom of the rack. -
Press Save.
-
Repeat steps 2 through 5 for each server in the rack. Each server boots up through BIOS, and boots the operating system with the default factory IP configuration.
Accessing a Server Using the KVM Switch
The following procedure describes how to access a server using the KVM switch:
Note:
The KVM switch is only available in Oracle Exadata Database Machine X2-2 racks and Oracle Exadata Storage Expansion Racks.-
Select Target Devices from Unit View.
-
Click the system name in the Name column using the left mouse button.
-
Click the KVM session.
Configuring Sun Datacenter InfiniBand Switch 36 Switch
The following procedure describes how to configure the Sun Datacenter InfiniBand Switch 36 switches.
-
Log in to the first database server as the
root
user. The first database server is the lowest database server in the rack, which is rack position U16.Note:
If you do not have the password for theroot
user, then contact Oracle Support Services. -
Use SSH to log in to the Sun Datacenter InfiniBand Switch 36 switch as the
root
user. The IP address for the Sun Datacenter InfiniBand Switch 36 switch is listed inChapter 3, "Understanding the Network Requirements for Oracle Exadata Database Machine." The following is an example of the command:ssh root@192.168.1.201
-
Use spsh to open the Integrated Lights Out Manager (ILOM) command line interface.
-
Configure the network interface using the following commands:
-
set /SP/network pendingipdiscovery=static pendingipaddress=pending_ip \ pendingipgateway=pending_gw pendingipnetmask=pending_nm set /SP/network commitpending=true
In the preceding commands, pending_ip, pending_gw, and pending_nm are IP addresses defined by the network administrator.
-
Use the
exit
command to exit the interface. -
Edit the
/etc/hosts
file to set the IP address and host name using an editor such as vi. The following is an example of the updated file:#Do not remove the following link, or various programs #that require network functionality will fail. 127.0.0.1 localhost.localdomain localhost 10.7.7.32 dm01sw-ib1-ib2.example.com trnasw-ib2
It is required that the second line has both fully-qualified and non-fully-qualified names.
-
Edit the
/etc/resolv.conf
file to set the DNS server and domain name using an editor such as vi. There should be a name server line for each DNS server. The following is an example of the updated file:search example.com nameserver 10.7.7.3
-
Edit the
/etc/sysconfig/network
file to include the host name. The following is an example of the syntax:hostname=switch_value
In the preceding syntax, switch_value is the fully-qualified domain name for the switch.
-
Use
spsh
to open the ILOM command line interface. -
Set the date, time zone, and Network Protocol Time (NTP) using the following commands:
set /SP/clock timezone=preferred_tz set /SP/clients/ntp/server/1 address=ntp_ip1 set /SP/clients/ntp/server/2 address=ntp_ip2 set /SP/clock usentpserver=enabled
In the preceding commands, preferred_tz is the preferred time zone, and ntp_ip1 and ntp_ip2 are the NTP server IP addresses. It is not necessary to configure both NTP servers, but at least one should be configured.
-
Restart the switch.
-
Examine the firmware version using the following command:
# version
See My Oracle Support Note 888828.1 for the current firmware version.
-
Check the health of the switch using the following command:
# showunhealthy OK - No unhealthy sensors
-
Run the environment test using the following command:
# env_test NM2 Environment test started: Starting Voltage test: Voltage ECB OK Measured 3.3V Main = 3.28 V Measured 3.3V Standby = 3.42 V Measured 12V =12.06 V Measured 5V =5.03 V Measured VBAT =3.06 V Measured 2.5V =2.53 V Measured 1.8V =1.79 V Measured I4 1.2V =1.22 V Voltage test returned OK Starting PSU test: PSU 0 present PSU 1 present PSU test returned OK Starting Temperature test: Back temperature 30.50 Front temperature 33.88 ComEx temperature 34.12 I4 temperature 56, maxtemperature 57 Temperature test returned OK Starting FAN test: Fan 0 not present Fan 1 running at rpm 12946 Fan 2 running at rpm 12684 Fan 3 running at rpm 12558 Fan 4 not present FAN test returned OK Starting Connector test: Connector test returned OK Starting I4 test: I4 OK All I4s OK I4 test returned OK NM2 Environment test PASSED
-
Enable the InfiniBand Subnet Manager using the following command:
# enablesm
Note:
If you get an error indicating the InfiniBand Subnet Manager is already running, then restart it as follows:# disablesm # enablesm
-
Verify the IP address is correct using the following command:
# ifconfig eth0 eth0 Link encap:Ethernet HWaddr 00:E0:4B:2A:07:2B inet addr:172.16.10.32 Bcast:172.16.10.255 Mask:255.255.255.0 inet6 addr:fe80::2e0:3a00:fe2a:61e/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:11927 errors:0 dropped:0 overruns:0 frame:0 TX packets:89 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:720262 (703.3 KiB)TX bytes:11402 (11.1 KiB)
-
Verify the host name is correct using the following command:
# hostname dm01sw-ib2.example.com
-
Set the Subnet Manager Master as described in "Setting the Subnet Manager Master on Exadata Database Machine Full Rack and Exadata Database Machine Half Rack". This step is needed for Exadata Database Machine Half Rack and Exadata Database Machine Full Rack.
-
Log out from the InfiniBand switch using the following command:
# exit
Note:
If the network settings did not appear after restart, then power cycle the switch by removing both power cords for one minute.Setting the Subnet Manager Master on Exadata Database Machine Full Rack and Exadata Database Machine Half Rack
Exadata Database Machine Full Racks and Oracle Exadata Database Machine X2-2 Half Racks have three Sun Datacenter InfiniBand Switch 36 switches. The switch at rack unit 1 (U1) is referred to as the spine switch. The switches at rack unit 20 (U20) and rack unit 24 (U24) in Oracle Exadata Database Machine X2-2 racks, or unit 21 (U21) and rack unit 23 (U23) in Oracle Exadata Database Machine X2-8 Full Racks are referred to as leaf switches. The spine switch is the Subnet Manager Master for the InfiniBand subnet. The Subnet Manager Master has priority 8, and can be verified using the following procedure:
See Also:
-
Log in to the spine switch as the
root
user. -
Run the
setsmpriority list
command.The command should show that
smpriority
has a value of 8. Ifsmpriority
has a different value, then do the following:-
Use the
disablesm
command to stop the Subnet Manager. -
Use the
setsmpriority 8
command to set the priority to 8. -
Use the
enablesm
command to restart the Subnet Manager.
-
The leaf switches are the Standby Subnet Managers with a priority of 5. This can be verified using the preceding procedure, substituting a value of 5 in the command shown in step 2b.
Note:
Exadata Database Machine Half Rack based on Sun Fire X4170 Oracle Database Servers include two Sun Datacenter InfiniBand Switch 36 switches, which are set to priority 5.To determine the Subnet Manager Master, log in as the root
user on any InfiniBand switch, and run the getmaster
command. The location of the Subnet Manager Master is displayed. The following is an example of the output from the getmaster
command:
# getmaster 20100701 11:46:38 OpenSM Master on Switch : 0x0021283a8516a0a0 ports 36 Sun DCS 36 QDR switch dm01sw-ib1.example.com enhanced port 0 lid 1 lmc 0
The preceding output shows the proper configuration. The Subnet Master Manager is running on spine switch dm01sw-ib1.example.com
.
If the spine switch is not the Subnet Manager Master, then do the following procedure to set the Subnet Manager Master:
-
Use the
getmaster
command to identify the current location of the Subnet Manager Master. -
Log in as the
root
user on the leaf switch that is the Subnet Manager Master. -
Disable the Subnet Manager on the switch. The Subnet Manager Master relocates to another switch.
See Also:
"Disable the Subnet Manager" in Sun Datacenter InfiniBand Switch 36 User's Guide athttp://docs.oracle.com/cd/E19197-01/835-0784-05/z4001de61813698.html#z40003f12047367
-
Use the
getmaster
command to identify the current location of the Subnet Manager Master. If the spine switch is not Subnet Manager Master, then repeat steps 2through 3 until the spine switch is the Subnet Manager Master. -
Enable the Subnet Manager on the leaf switches that were disabled during this procedure.
See Also:
"Enable the Subnet Manager" in Sun Datacenter InfiniBand Switch 36 User's Guide athttp://docs.oracle.com/cd/E19197-01/835-0784-05/z4001de61707660.html#z40003f12047359
Notes:
-
If the InfiniBand network consists of four or more racks cabled together, then only the spine switches should run Subnet Manager. The leaf switches should have Subnet Manager disabled on them.
-
Exadata Database Machine Half Racks based on Sun Fire X4170 Oracle Database Servers and Exadata Database Machine Quarter Racks have two Sun Datacenter InfiniBand Switch 36 switches, and both are set to priority 5. The master is the one with the lowest GUID.
Configuring the Cisco Ethernet Switch
The Cisco Catalyst 4948 Ethernet switch supplied with Oracle Exadata Rack is minimally configured during installation. The minimal configuration disables IP routing, and sets the following:
-
Host name
-
IP address
-
Subnet mask
-
Default gateway
-
Domain name
-
Name server
-
NTP server
-
Time
-
Time zone
Before configuring the switch, note the following:
-
The Cisco Ethernet switch should not be connected until the running configuration has been verified, and any necessary changes have been made by the network administrator.
-
The Cisco Ethernet switch should not be connected to the customer network until the IP addresses on all components have been configured in Oracle Exadata Rack. This is to prevent any duplicate IP address conflicts which are possible due to the default addresses set in the components when shipped.
See Also:
The following procedure describes how to configure the Cisco Ethernet switch. Configuration should be done with the network administrator.
-
Connect a serial cable from the Cisco switch console to a laptop or similar device. An RJ45 to DB9 serial cable is included with the Cisco documentation package.
-
Ensure the terminal session is recorded on the laptop by logging the output. The output can be used as a reference that the switch has been configured correctly. Thedefault serial port speed is 9600 baud, 8 bits, no parity, 1 stop bit, and no handshake.
Switch con0 is now available Press RETURN to get started.
-
Change to enable mode using the following command:
Switch> enable
Note:
If you do not have the password for theroot
user, then contact Oracle Support Services. -
Configure the network for a single VLAN. The following is an example of the configuration:
Switch# configure terminal Enter configuration commands,one per line.End with CNTL/Z. Switch(config)# interface vlan 1 Switch(config-if)# ip address 10.7.7.34 255.255.255.0 Switch(config-if)# end Switch# *Sep 15 14:12:06.309:%SYS-5-CONFIG_I:Configured from console by console Switch# write memory Building configuration... Compressed configuration from 2474 bytes to 1066 bytes [OK ]
-
(Optional) Disable the default IP routing setting, and configure the default gateway as follows. This step is required if IP routing will not be used on the switch.
Switch# configure terminal Enter configuration commands,one per line.End with CNTL/Z. Switch(config)# no ip routing Switch(config)# ip default-gateway 10.7.7.1 Switch(config)# end *Sep 15 14:12:46.309:%SYS-5-CONFIG_I:Configured from console by console Switch# write memory Building configuration... Compressed configuration from 2492 bytes to 1070 bytes [OK ]
Note:
If IP routing is required on the switch, then leave the IP routing setting as the default, and configure the default gateway as follows:Switch# configure terminal Enter configuration commands,one per line.End with CNTL/Z. Switch(config)# ip route 0.0.0.0 0.0.0.0 10.7.7.1 Switch(config)# end *Sep 15 14:13:26.013:%SYS-5-CONFIG_I:Configured from console by console Switch# write memory Building configuration... Compressed configuration from 2502 bytes to 1085 bytes [OK ]
-
Set the host name of the switch as follows:
Switch# configure terminal Enter configuration commands,one per line.End with CNTL/Z. Switch(config)# hostname burxsw-ip burxsw-ip(config)# end burxsw-ip# write memory Building configuration... Compressed configuration from 3789 bytes to 1469 bytes [OK ]
The system host name is used as the prompt name.
-
Set the password as follows:
burxsw-ip # configure terminal Enter configuration commands,one per line.End with CNTL/Z. burxsw-ip(config)# enable password ******* burxsw-ip(config)# enable secret ******* The enable secret you have chosen is the same as your enable password. This is not recommended.Re-enter the enable secret. burxsw-ip(config)# end burxsw-ip#write memory *Sep 15 14:25:05.893:%SYS-5-CONFIG_I:Configured from console by console Building configuration... Compressed configuration from 2502 bytes to 1085 bytes [OK ]
-
Set the password for telnet network access. The following is an example:
Switch# configure terminal Enter configuration commands,one per line.End with CNTL/Z. burxsw-ip(config)# line vty 0 15 burxsw-ip(config-line)# login %Login disabled on line 1,until 'password'is set %Login disabled on line 2,until 'password'is set %Login disabled on line 3,until 'password'is set ... %Login disabled on line 15,until 'password'is set %Login disabled on line 16,until 'password'is set burxsw-ip(config-line)# password ******* burxsw-ip(config-line)# login burxsw-ip(config-line)# end burxsw-ip#write memory Building configuration... Compressed configuration from 3786 bytes to 1468 bytes [OK ]
In the preceding example, the first
login
output shows the password has not been set, and telnet access is disabled. If thelogin
command returns nothing, then the password is set and telnet access is available. -
Configure up to three DNS servers. The following is an example:
burxsw-ip# configure terminal Enter configuration commands,one per line.End with CNTL/Z. burxsw-ip(config)# ip domain-name exdm.com burxsw-ip(config)# ip name-server 10.7.7.3 burxsw-ip(config)# ip name-server 129.148.5.5 burxsw-ip(config)# ip name-server 10.8.160.1 burxsw-ip(config)# end *Sep 15 14:26:37.045:%SYS-5-CONFIG_I:Configured from console by console burxsw-ip#write memory Building configuration... Compressed configuration from 2603 bytes to 1158 bytes [OK ]
-
Set the clock and time zone as follows. The switch keeps internal time in Coordinated Universal Time (UTC) format.
-
To use UTC, use the following command:
no clock timezone global configuration
-
To use a time zone, use the following command:
clock timezone zone hours-offset [minutes-offset]
In the preceding command, zone is the time zone to display when standard time in effect, hours-offset is the hours offset from UTC, and minutes-offset is the minutes offset from UTC.
-
To set summer time hours, use the following command:
clock summer-time zone recurring [week day month hh:mm week day month \ hh:mm [offset]]
In the preceding command, zone is the time zone to be displayed when summer time is in effect, week is the week of the month (1 to 5 or last), day is the day of the week, month is the month, hh:mm is the time in 24-hour format, and offset is the number of minutes to add during summer time. The default offset is 60 minutes.
-
To manually set the clock to any time use the following command:
clock set hh:mm:ss month day year
In the preceding command, hh:mm:ss is the time in 24-hour format, day is the day of the month, month is the month, and year is the year. The time specified is relative to the configured time zone.
To set the local time and time zone, ordering is important. The following is an example of setting local time to US Eastern time:
Switch# configure terminal Enter configuration commands,one per line.End with CNTL/Z. burxsw-ip(config)# clock timezone EST -5 burxsw-ip(config)# clock summer-time EDT recurring burxsw-ip(config)# end burxsw-ip# clock set 21:00:00 August 09 2010 burxsw-ip#write memory Building configuration... Compressed configuration from 3784 bytes to 1465 bytes [OK ] burxsw-ip# show clock 21:00:06.643 EST Mon Aug 9 2010
-
-
Configure up to two NTP servers. The following is an example:
burxsw-ip# configure terminal Enter configuration commands,one per line.End with CNTL/Z. burxsw-ip(config)# ntp server 10.7.7.32 prefer burxsw-ip(config)# ntp server 129.148.9.19 burxsw-ip(config)# end *Sep 15 14:51:08.665:%SYS-5-CONFIG_I:Configured from console by console burxsw-ip# write memory Building configuration... Compressed configuration from 2654 bytes to 1163 bytes [OK ] burxsw-ip# show ntp status <output will vary per network> burxsw-ip# show clock 20:59:06.643 EST Mon Aug 9 2010
The preceding should show the NTP server synchronized to local time if the Cisco switch is connected to the network and has access to NTP.
-
Verify the configuration using the following command:
burxsw-ip# show running-config
The following is an example of the output:
Building configuration... Current configuration :2654 bytes ! version 12.2 no service pad service timestamps debug datetime msec service timestamps log datetime msec no service password-encryption service compress-config ! hostname Switch ! boot-start-marker boot-end-marker ! enable secret 5 $1$mS8h$EaJrIECUxavfGH6vLZg1T. enable password ******* ! no aaa new-model ip subnet-zero ip domain-name sodm.com ip name-server 10.7.7.3 ip name-server 172.16.5.5 ip name-server 10.8.160.1 ! ip vrf mgmtVrf ! vtp mode transparent ! power redundancy-mode redundant ! ! spanning-tree mode pvst spanning-tree extend system-id ! vlan internal allocation policy ascending ! ! interface FastEthernet1 ip vrf forwarding mgmtVrf no ip address speed auto duplex auto ! interface GigabitEthernet1/1 spanning-tree portfast ! interface GigabitEthernet1/2 spanning-tree portfast ! ... ! interface GigabitEthernet1/47 media-type rj45 spanning-tree portfast ! interface GigabitEthernet1/48 media-type rj45 ! interface Vlan1 ip address 10.7.7.34 255.255.255.0 ! interface Vlan48 no ip address shutdown ! ip default-gateway 10.7.7.1 ip http server ! ! control-plane ! ! line con 0 stopbits 1 line vty 0 4 password ******* login line vty 5 15 password ******* login ! ntp server 10.7.7.32 prefer ntp server 172.16.9.1 end
If any setting is incorrect, the repeat the appropriate step. To erase a setting, enter
no
in front of the same command. For example, to erase the default gateway, the following commands would be entered:burxsw-ip# no ip default-gateway 10.7.7.1 burxsw-ip# end burxsw-ip# write memory
-
Save the current configuration using the following command:
burxsw-ip# copy running-config startup-config
-
Exit from the session using the following command:
burxsw-ip# exit
-
Disconnect the cable from the Cisco console.
-
Attach a laptop computer to port 48, and ping the IP address of the internal management network to check the configuration.
Configuring the Power Distribution Units
The power distribution units (PDUs) are configured with a static IP address to connect to the network for monitoring. Ensure you have the following before connecting the PDU to the network:
-
Static IP address
-
Subnet mask
-
Default gateway
-
Ethernet cables
-
Laptop computer with either Oracle Solaris or Microsoft Windows
The following procedure describes how to configure and connect the PDU to the network:
-
Power off the PDU.
-
Disconnect the power leads from the power source.
-
Select an unused LAN network connection from the available network connections on the laptop as follows:
-
For Oracle Solaris:
-
Log in as a superuser.
-
Use the
dladm show-link
command to find an unused Ethernet interface. -
Use the
ifconfig -a
command to determine which interface is being used. -
Use the following command to plumb the unused interface:
# ifconfig interface plumb up
In the preceding command, interface is the network interface determined in step c.
-
Using the following command to assign an IPv4 address and netmask to the interface:
# ifconfig interface IPv4_address netmask + netmask \ broadcast broadcast up
In the preceding command, interface is the network interface, IPv4_address is the IP address, netmask is the netmask address, and broadcast is the broadcast address.
Note:
The network interfaces configured with theifconfig
command do not persist across system restarts.
-
-
For Windows:
-
Select Settings from the Start menu.
-
Select Network Connections from the Settings menu.
-
Select an unused local area connection from the Network Connections menu.
-
Double-click Internet Protocol (TCP/IP) in the Local Area Connection Properties window. The Internet Protocol (TCP/IP) window appears.
-
Select Use the following IP address.
-
Enter the IP address in the IP address field. Do not use 192.168.0.1 because that is the default PDU metering unit address.
-
Enter the IP address in the Subnet mask field.
-
Leave the Default gateway field blank.
-
Click OK.
-
-
-
Connect the PDU and the laptop to two ports in the Cisco switch. The equipment currently using the Cisco switch must be disconnected temporarily.
-
Connect the PDU power input lead to the power source. Only one PDU power input lead needs to be connected, as follows:
-
On PDUs with one power input lead, connect the one lead to the power source.
-
On PDUs with two power leads, connect the first power lead to the power source. The first power lead is labelled 0.
-
On PDUs with three power input leads, connect the middle power lead to the power source. The middle power lead is labelled 1.
-
-
Use a Web browser to access the PDU metering unit by entering the factory default IP address for the unit,
192.168.0.1
, in the address line of the browser. The Current Measurement page should appear. -
Click Network Configuration in the upper left of the page.
-
Log in as the
admin
user on the PDU metering unit. Change the password after configuring the network. -
Confirm the DHCP Enable check box is not selected.
-
Enter the following network settings for the PDU metering unit:
-
IP address
-
Subnet mask address
-
Default gateway
-
-
Click Submit to set the network settings, and reset the PDU metering unit.
-
Disconnect the PDU and laptop from the Cisco switch.
-
Reconnect the two cables that were originally connected to the Cisco switch.
-
Connect an Ethernet cable to the PDU metering unit RJ-45 Ethernet port and to the network.
-
Log in to the PDU metering unit using a Web browser. Use the PDU metering unit's static IP address in the browser's address line. If the configuration was successful, then the Current Measurement page is displayed.
-
Connect the remaining PDU power input leads to the power source.
Checking Exadata Storage Servers
The following procedure describes how to check Exadata Storage Servers:
Notes:
-
When using the KVM switch, and pressing the ESC key, the BIOS may receive two ESC characters, and prompt to exit. SelectCANCEL.
-
If you need to connect to the ILOM serial management port, then the baud rate setting on Exadata Storage Servers is changed from the default ILOM setting 9600 to 115200 baud, 8 bits, no parity, and 1 stop bit.
-
Log in as the
root
user to the first Exadata Storage Server.Note:
If you do not have the password for theroot
user, then contact Oracle Support Services. -
Verify the SysSN label for the cell matches the
product_serial_number
value using the following command:#ipmitool sunoem cli "show /SYS product_serial_number" Connected.Use ^ D to exit. ->show /SYS product_serial_number /SYS Properties: product_serial_number =0937XFG03B ->Session closed Disconnected
-
Check the hardware and firmware using the following command:
# /opt/oracle.SupportTools/CheckHWnFWProfile
If output does not show success, then examine the output for mismatched hardware or firmware and take steps to correct those as explained in next steps in this section.
-
Verify the disks on the server are visible and online using the following command. The disks are numbered slot 0 to slot 11.
#cd /opt/MegaRAID/MegaCli #./MegaCli 64 -Pdlist -aAll |grep "Slot \|Firmware" Slot Number:0 Firmware state:Online, Spun Up Slot Number:1 Firmware state:Online, Spun Up Slot Number:2 Firmware state:Online, Spun Up Slot Number:3 Firmware state:Online, Spun Up Slot Number:4 Firmware state:Online, Spun Up Slot Number:5 Firmware state:Online, Spun Up
-
Slot Number:6 Firmware state:Online, Spun Up Slot Number:7 Firmware state:Online, Spun Up Slot Number:8 Firmware state:Online, Spun Up Slot Number:9 Firmware state:Online, Spun Up Slot Number:10 Firmware state:Online, Spun Up Slot Number:11 Firmware state:Online, Spun Up [root@cell01 ~]#
-
Verify the flash modules are visible using the following command. There are four cards, and four modules per card.
# lsscsi |grep -i marvel # cellcli -e list lun where disktype=flashdisk
The numbering shows the PCI slot number and FMOD number, such as
1_0
is PCI slot 1, FMOD 0 on the card. To confirm the flash disks are mapped that way, use the following command:# cellcli -e list physicaldisk attributes name, id, slotnumber where \ disktype="flashdisk" and 'status! ="not present"'
If any FMODs are missing, then power cycle the system, and wait 10 minutes after booting to check. If this does not resolve the problem, then reseat the module and flash card. If the problem continues, then replace the card with the onsite spare.
-
Check the rear of the rack to ensure the flash card LEDs are green all the way across. These are FMOD, 0, 1, 2, 3, and SuperCAP. If any LED is amber, then check again after the next step. If the LED is still amber after restart, then reseat the card and ESM or FMOD. If the problem continues, then replace the card with the onsite spare.
Note:
This step can be done when all systems are on. It does not have to be done on a per-server basis. -
Log out of the system using the following command:
# logout
-
Return to the User Interface on the KVM switch by pressing the Ctrl key on the left to return to the KVM switch, and select Disconnect Session, if using the KVM switch to access the server.
-
Select Target Devices from Unit View, if using the KVM switch to access the server.
-
Proceed to the next Exadata Storage Server.
Checking Oracle Database Servers
The following procedure describes how to check the Oracle Database servers. At this point, the operating system is Linux.
-
Log in as the
root
user to the first database server.Note:
If you do not have the password for theroot
user, then contact Oracle Support Services. -
Verify the SysSN label for the server matches the
product_serial_number
value using the following command:# ipmitool sunoem cli "show /SYS product_serial_number" Connected.Use ^ D to exit. ->show /SYS product_serial_number /SYS Properties: product_serial_number =093ABCD5001 ->Session closed Disconnected
-
Verify the hardware profile is correct using the following command. The command checks CPU type and count, component firmware, and so on.
# dcli -l root -g half "/opt/oracle.SupportTools/CheckHWnFWProfile -c strict" > /tmp/checkhwfw.out # more /tmp/checkhwfw.out
The following is an example of a check that failed:
172.108.1.6:[WARNING ]The hardware and firmware are not supported. See details below [DiskControllerPCIeSlotWidth ] Requires: x8 Found: x4 [WARNING ]The hardware and firmware are not supported.See details above
If a profile check fails and the firmware needs to be refreshed, then run the following command from the system with the problem.
# /opt/oracle.SupportTools/CheckHWnFWProfile -U /opt/oracle.cellos/iso/cellbits Now updating the ILOM and the BIOS …
The command must be entered on a single line. After running the command, power off the system, reset the ILOM, and wait 10 minutes before powering on the system.
-
Verify the disks on the server are visible and online using the following command. All but one disk should show
Spun Up
. The one disk that showsSpun Down
is the hot spare. For Oracle Exadata Database Machine X2-2, there are four disks, and for Oracle Exadata Database Machine X2-8 Full Rack, there are eight disks.# cd /opt/MegaRAID/MegaCli # ./MegaCli 64 -Pdlist -aAll |grep "Slot \|Firmware" Firmware state:Online, Spun Up Slot Number:1 Firmware state:Online, Spun Up Slot Number:2 Firmware state:Online, Spun Up Slot Number:3 Firmware state:Hotspare, Spun Down
-
Examine the output of the following command:
/opt/MegaRAID/MegaCli/MegaCli64 -LdInfo -LAll -aAll
For Oracle Exadata Database Machine X2-2, the shipped configuration is dual boot. Verify there is one RAID-1 and two RAID-0 logical (virtual) drives.
For Oracle Exadata Database Machine X2-8 Full Rack, the shipped configuration is dual boot. There should be one 3-disk RAID-5 and two 2-disk RAID-1 logical drives with one global hot spare drive.
-
(Oracle Exadata Database Machine based on Sun Fire X4170 Oracle Database Servers only) Check that the ILOM is configured for the proper fan cooling on each database server using the following command:
# ipmitool sunoem cli "show /SP/policy"
-
If the policy is present and enabled, then no additional steps are needed.
-
If the policy is present and disabled, then enable the policy using the following command:
# ipmitool sumoem cli "set ?SP/policy FLASH_ACCELERATOR_CARD_ \ INSTALLED=enabled"
The command must be entered as a single line.
-
-
Log out of the system using the following command:
# logout
-
Return to the User Interface on the KVM switch by pressing the Ctrl key on the left to return to the KVM switch, and select Disconnect Session, if using the KVM switch to access the server.
-
Select Target Devices from Unit View, and proceed to the next database server, if using the KVM switch to access the server.
-
Verify that NTP servers are configured the same for all servers using the following command:
dcli -g all_group -l root "ntpq -p"
Performing Additional Checks and Configuration
The following procedure describes additional checks and configuration steps for Oracle Exadata Rack:
-
(Oracle Exadata Database Machine only) Log in to the first database server as the
root
user. -
Change to the
firstconf
directory using the following command:# cd /opt/oracle.SupportTools/firstconf
-
Verify the network port for eth0 can be accessed over the Cisco switch using the following command:
# ./ping_check_ilom_and_eth0.sh rack_size
In the preceding command, rack_size is the size of the rack. For Oracle Exadata Database Machine X2-2 Full Rack use
full
, for Oracle Exadata Database Machine X2-2 Half Rack usehalf
, for Oracle Exadata Database Machine X2-2 Quarter Rack usequarter
, and for Oracle Exadata Database Machine X2-8 Full Rack usefull_v2pp
. The following is an example of a failure:# ./ping_check_ilom_and_eth0.sh full_v2pp [INFO ]Checking nodes are pingable... [ERROR ]Following nodes are not pingable.Check connections and retry. Unpingable:172.16.10.25 172.16.10.18
A failure indicates a bad connection, faulty cable, bad network port, or bad switch port.
-
Verify the NTP servers are configured the same for all servers using the following command:
dcli -g all_group -l root "ntpq -p"
-
Verify the rack master serial number is set correctly using the following command:
# dcli -l root -g half "ipmitool sunoem cli 'show /SP system_identifier'" > /tmp/show-rack-csn.out # more /tmp/show-rack-csn.out
If none of the serial numbers are set, then contact Oracle Support Services. If one serial number is not set correctly, then run the following command to correct it:
ipmitool sunoem cli 'set /SP system_identifier= \ "Exadata Database Machine rack_typexxxxAKyyyy"'
In the preceding command, rack_type is
X2-2
orX2-8
, depending on the rack. -
Ensure disk cache policy is set to
Disabled
using the following command as theroot
user on the first database server in the cluster:dcli -g /opt/oracle.SupportTools/onecommand/all_group -l root \ /opt/MegaRAID/MegaCli/MegaCli64 -LdPdInfo -aALL | grep -i 'Disk Cache Policy'
The following is an example of the output from the command:
dm01db01: Disk Cache Policy : Disabled dm01db02: Disk Cache Policy : Disabled dm01db03: Disk Cache Policy : Disabled ... dm01cel01: Disk Cache Policy : Disabled dm01cel02: Disk Cache Policy : Disabled ...
If any server shows different output, then identify the LUN and use the following command to reset the policy to
Disabled
:MegaCli64 -LDSetProp -DisDskCache -LUNn -a0
In the preceding command, LUNn is the LUN number, such as L2.
Verifying the InfiniBand Network
The following procedure describes how to verify the InfiniBand network:
-
Visually check all the InfiniBand cable connections within the rack. The port lights should be on, and the LEDs should be on. Do not press each connector to verify connectivity.
-
Log in as the
root
user on any component in the rack. -
Verify the InfiniBand topology using the following commands:
# cd /opt/oracle.SupportTools/idiagtools # ./verify-topology [-t rack_size]
The following example shows the output when the network components are correct.
[DB Machine Infiniband Cabling Topology Verification Tool ] Is every external switch connected to every internal switch......[SUCCESS ] Are any external switches connected to each other................[SUCCESS ] Are any hosts connected to spine switch..........................[SUCCESS ] Check if all hosts have 2 CAs to different switches..............[SUCCESS ] Leaf switch check:cardinality and even distribution..............[SUCCESS ] Check if each rack has an valid internal ring....................[SUCCESS ]
In the preceding command, rack_size is the size of the rack. The
-t
rack_size
option is needed if the rack is Exadata Database Machine Half Rack or Exadata Database Machine Quarter Rack. Usehalfrack
for Exadata Database Machine Half Rack, andquarterrack
for Exadata Database Machine Quarter Rack.The following example shows the output when there is a bad InfiniBand switch to InfiniBand cable connection:
#./verify-topology [DB Machine Infiniband Cabling Topology Verification Tool ] Is every external switch connected to every internal switch......[SUCCESS ] Are any external switches connected to each other................[SUCCESS ] Are any hosts connected to spine switch..........................[SUCCESS ] Check if all hosts have 2 CAs to different switches..............[SUCCESS ] Leaf switch check:cardinality and even distribution..............[SUCCESS ] Check if each rack has an valid internal ring....................[ERROR ] Switches 0x21283a87cba0a0 0x21283a87b8a0a0 have 6 connections between them. They should have at least 7 links between them
The following example shows the output when there is a bad InfiniBand cable on a database server:
#./verify-topology [DB Machine Infiniband Cabling Topology Verification Tool ] Is every external switch connected to every internal switch......[SUCCESS ] Are any external switches connected to each other................[SUCCESS ] Are any hosts connected to spine switch..........................[SUCCESS ] Check if all hosts have 2 CAs to different switches..............[ERROR ] Node db01 has 1 endpoints.(Should be 2) Port 2 of this node is not connected to any switch --------fattree End Point Cabling verification failed----- Leaf switch check:cardinality and even distribution..............[ERROR ] Internal QDR Switch 0x21283a87b8a0a0 has fewer than 4 compute nodes It has only 3 links belonging to compute nodes [SUCCESS ] Check if each rack has an valid internal ring...................[SUCCESS ]
The following example shows the output when there is a bad connection on the switch and the system:
#./verify-topology [DB Machine Infiniband Cabling Topology Verification Tool ] Is every external switch connected to every internal switch......[SUCCESS ] Are any external switches connected to each other................[SUCCESS ] Are any hosts connected to spine switch..........................[SUCCESS ] Check if all hosts have 2 CAs to different switches..............[ERROR ] Node burxdb01 has 1 endpoints.(Should be 2) Port 2 of this node is not connected to any switch --------fattree End Point Cabling verifation failed----- Leaf switch check:cardinality and even distribution..............[ERROR ] Internal QDR Switch 0x21283a87b8a0a0 has fewer than 4 compute nodes It has only 3 links belonging to compute nodes...................[SUCCESS ] Check if each rack has an valid internal ring....................[ERROR ] Switches 0x21283a87cba0a0 0x21283a87b8a0a0 have 6 connections between them They should have at least 7 links between them
Selecting the Operating System
Oracle Exadata Database Machine is shipped with the Linux operating system and Oracle Solaris operating system for the Oracle Database servers. Linux is the default operating system. The following procedure describes how to select Oracle Solaris as the operating system:
-
Log in as the
root
user. -
Change to the
/opt/SupportTools
directory. -
Run the following command:
defaultOSchoose.pl
The script will prompt as follows:
Default OS is : LINUX_BOOT_0 Please choose new default OS: [0] LINUX_BOOT_0 [1] SOLARIS_BOOT_1 [2] SOLARIS_BOOT_2 Please type the number you would like to make a new default OS:
-
Enter
1
to select SOLARIS_BOOT_1 for the operating system, and press Enter.
Reclaiming Disk Space After Selecting the Operating System
After selecting an operating system, reclaim the disk space occupied by the other operating system. The disk space is shipped in the following configurations:
-
The four-disk database servers in Oracle Exadata Database Machine X2-2 are shipped with Linux using first two disks in hardware RAID-1 with RAID-1 implemented as the disk controller configuration, and the second two disks used by Oracle Solaris in a ZFS RAID-1 configuration.
-
For the eight-disk database servers in Oracle Exadata Database Machine X2-8 Full Rack, Linux resides on first four disks configured as three disks in RAID-5 at the disk controller level with the fourth disk as the global hot spare. Oracle Solaris uses the other four disks with two disks making one ZFS RAID-1 for root pool, and two disks making one ZFS RAID-1 for the data pool.
The following sections describe how to reclaim the disk space:
Caution:
Do not restart Oracle Exadata Database Machine or interrupt the reclamation process.Reclaiming Disks for the Linux Operating System
Reclaiming the disks on a four-disk database server with Linux as the operating system, converts the system to have three disks RAID-5 with the fourth disk as the hot spare at the disk controller. Reclaiming the disks on an eight-disk database server with Linux as the operating system, creates seven-disk RAID-5 with the eighth disk as the hot spare at the disk controller level.
The following procedure describes how reclaim the disk space used by the Oracle Solaris operating system. This procedure should be performed on each database server.
-
Log in as the
root
user. -
Change to the
/opt/oracle.SupportTools
directory. -
Check the current disk configuration using the following command:
./reclaimdisks.sh -check
The command returns a detailed layout of the logical and physical disks. For Oracle Exadata Database Machine X2-2, the last line of the output should be the following:
[INFO] Valid dual boot configuration found for Linux: RAID1 from 2 disks
For Oracle Exadata Database Machine X2-8 Full Rack, the last line of the output should be the following:
[INFO] Valid dual boot configuration found for Linux: RAID5 from 3 disks and 1 global hot spare disk
-
Start the disk reclamation process using the following command:
./reclaimdisks.sh -free -reclaim
The command frees any Oracle Solaris-configured disks, and reclaims all free disks for Linux. The process may take two hours for Oracle Exadata Database Machine X2-2, and five hours for Oracle Exadata Database Machine X2-8 Full Rack. To check the progress of the reclamation process, use the following command:
tail -f /var/log/cellos/reclaimdisks.bg.log
-
Run the following command when the disk reclamation process is complete:
./reclaimdisks.sh -check
For Oracle Exadata Database Machine X2-2, the last line of the output should be the following:
INFO] Valid single boot configuration found for Linux: RAID5 from 3 disks and 1 global hot spare disk
For Oracle Exadata Database Machine X2-8 Full Rack, the last line of the output should be the following:
[INFO] Valid single boot configuration found for Linux: RAID5 from 7 disks and 1 global hot spare disk
Reclaiming Disks for the Oracle Solaris Operating System
For four-disk database servers with Oracle Solaris as the operating system, reclaiming disks creates a data pool with two reclaimed disks in ZFS RAID-1 configuration. For the eight-disk database server with Oracle Solaris as the operating system, reclaiming disks results in two ZFS RAID-1 sets created from the reclaimed disks to the existing data pool. The /opt/oracle.SupportTools/disk_map.pl
script on Oracle Solaris can assist in mapping the physical enclosure:slot location of disks to the device names, ZFS pools and the mount points for the pools.
The following procedure describes how reclaim the disk space used by the Linux operating system. This procedure should be performed on each database server.
-
Log in as the
root
user on the database server. -
Change to the
/opt/SupportTools
directory. -
Install the latest reclaimdisks.pl script from patch 12854525.
-
Run the
reclaimdisks.pl
script as follows:reclaimdisks.pl [--unattended]
The script runs in interactive mode. To run the script in unattended mode, use the
--unattended
option.Note:
In some cases, the system is unable to restart because the boot disks are not marked as such in the LSI controller. If this occurs, then do one of the following:-
Boot in single user mode using the Oracle Solaris flash drive, and run the following command:
# /opt/MegaRAID/MegaCLI - AdpBootDrive set L1 -a0
-
Use the LSI preboot command line interface as follows:
-
Press CTRL+Y while powering the database server. This brings up the LSI preboot interface.
-
Run the following command at the prompt:
$ -adpbootdrive set l1 -a0
-
-
-
Respond to the prompts, as needed. The script prompts for confirmation to remove the Linux virtual disks, and to create Oracle Solaris virtual disks.
The following is an example of the output when running the script on Oracle Exadata Database Machine X2-8 Full Rack:
This script changes the Linux virtual and physical disks to Solaris disks.
The system has identified unused virtual and physical disks.
The results are as follows:
Unused Virtual Disks:
/opt/MegaRAID/MegaCli -LDInfo -L0 -a0 Virtual Drive: 0 (Target Id: 0)
Name :DBSYS
RAID Level : Primary-5, Secondary-0, RAID Level Qualifier-3
Size : 557.75 GB
State : Optimal
Stripe Size : 1.0 MB
Number Of Drives : 3
Span Depth : 1
Default Cache Policy: WriteBack, ReadAheadNone, Direct, No Write
Cache if Bad BBU
Current Cache Policy: WriteBack, ReadAheadNone, Direct, No Write
Cache if Bad BBU
Access Policy : Read/Write
Disk Cache Policy : Disabled
Encryption Type : None
Number of Dedicated Hot Spares: 1
0 : EnclId - 17 SlotId - 3
The virtual disks will be deleted. Do you want to proceed? (yes/no) yes
This is an irreversible operation. Confirm that you want
to proceed. (yes/no) yes
Deleting the virtual disks ...
/opt/MegaRAID/MegaCli -CfgLdDel -L0 -a0
Adapter 0: Deleted Virtual Drive-0(target id-0)
Done.
Disabling hot spare disks ...
/opt/MegaRAID/MegaCli -pdHSP -Rmv -PhysDrv[17:3] -a0
Adapter: 0: Remove Physical Drive at EnclId-17 SlotId-3 as Hot Spare Success.
Unused Physical Drives:
/opt/MegaRAID/MegaCli -pdInfo -PhysDrv[17:3] -a0
Slot Number: 3
Device Id: 14
PD Type: SAS
Raw Size: 279.396 GB [0x22ecb25c Sectors]
Inquiry Data: SEAGATE ST930003SSUN300G0868102570CHPY
/opt/MegaRAID/MegaCli -pdInfo -PhysDrv[17:0] -a0
Slot Number: 0
...
New Virtual Drives will be created from these Physical Disks. Do you want
to proceed? (yes/no) yes
/opt/MegaRAID/MegaCli -CfgLdAdd -R0[17:3] WB NORA Direct NoCachedBadBBU
-strpsz1024 -a0
Adapter 0: Created VD 0
Adapter 0: Configured the Adapter!!
/opt/MegaRAID/MegaCli -CfgLdAdd -R0[17:0] WB NORA Direct NoCachedBadBBU
-strpsz1024 -a0
...
Adapter 0: Created VD 7
Adapter 0: Configured the Adapter!!
There are 4 disks from which 'data' pool may be created (or extended)
Do you want to proceed? (yes/no) yes
Creating an additional group in 'data' pool
zpool add -f data mirror c2t7d0 c2t6d0
Creating an additional group in 'data' pool
zpool add -f data mirror c2t5d0 c2t0d0
The disks reconfiguration has completed
The following is sample output for Oracle Exadata Database Machine X2-2 before reclaiming disks:
zpool list NAME SIZE ALLOC FREE CAP DEDUP HEALTH ALTROOT rpool 278G 42.9G 235G 15% 1.00x ONLINE - /opt/oracle.SupportTools/disks_map.pl ==================================================== LOCATION DISK NAME POOL NAME ==================================================== ------------------- mirror-0 --------------------- 252:3 c1t2d0s0 rpool 252:2 c1t1d0s0 rpool -------------------------------------------------- ==================================================== POOL/FILESYSTEM MOUNTPOIT ==================================================== rpool /rpool rpool/ROOT legacy rpool/ROOT/solaris /a rpool/dump - rpool/export /export rpool/export/home /export/home rpool/savecore /var/crash/solaris rpool/swap - rpool/u01 /u01
The following is sample output for Oracle Exadata Database Machine X2-2 after reclaiming disks:
zpool list NAME SIZE ALLOC FREE CAP DEDUP HEALTH ALTROOT data 278G 9.34G 269G 3% 1.00x ONLINE - rpool 278G 33.6G 244G 12% 1.00x ONLINE - /opt/oracle.SupportTools/disks_map.pl ==================================================== LOCATION DISK NAME POOL NAME ==================================================== ------------------- mirror-0 --------------------- 252:3 c1t2d0s0 rpool 252:2 c1t1d0s0 rpool -------------------------------------------------- ------------------- mirror-0 --------------------- 252:0 c1t0d0 data 252:1 c1t3d0 data -------------------------------------------------- ==================================================== POOL/FILESYSTEM MOUNTPOIT ==================================================== data /data data/u01 /u01 rpool /rpool rpool/ROOT legacy rpool/ROOT/solaris /a rpool/dump - rpool/export /export rpool/export/home /export/home rpool/savecore /var/crash/solaris rpool/swap
The following is sample output for Oracle Exadata Database Machine X2-8 Full Rack before reclaiming disks:
zpool list NAME SIZE ALLOC FREE CAP DEDUP HEALTH ALTROOT data 278G 9.29G 269G 3% 1.00x ONLINE - rpool 278G 39.7G 238G 14% 1.00x ONLINE - /opt/oracle.SupportTools/disks_map.pl ==================================================== LOCATION DISK NAME POOL NAME ==================================================== ------------------- mirror-0 --------------------- 16:5 c1t2d0s0 rpool 16:4 c1t1d0s0 rpool -------------------------------------------------- ------------------- mirror-0 --------------------- 16:6 c1t3d0 data 16:7 c1t4d0 data -------------------------------------------------- ==================================================== POOL/FILESYSTEM MOUNTPOIT ==================================================== data /data data/u01 /u01 rpool /rpool rpool/ROOT legacy rpool/ROOT/solaris /a rpool/dump - rpool/export /export rpool/export/home /export/home rpool/savecore /var/crash/solaris rpool/swap -
The following is sample output for Oracle Exadata Database Machine X2-8 Full Rack after reclaiming disks:
zpool list NAME SIZE ALLOC FREE CAP DEDUP HEALTH ALTROOT data 834G 9.14G 825G 1% 1.00x ONLINE - rpool 278G 33.6G 244G 12% 1.00x ONLINE - /opt/oracle.SupportTools/disks_map.pl ==================================================== LOCATION DISK NAME POOL NAME ==================================================== ------------------- mirror-0 --------------------- 16:5 c1t2d0s0 rpool 16:4 c1t1d0s0 rpool -------------------------------------------------- ------------------- mirror-0 --------------------- 16:6 c1t3d0 data 16:7 c1t4d0 data -------------------------------------------------- ------------------- mirror-2 --------------------- 16:0 c1t5d0 data 16:2 c1t6d0 data -------------------------------------------------- ------------------- mirror-1 --------------------- 16:3 c1t0d0 data 16:1 c1t7d0 data -------------------------------------------------- ==================================================== POOL/FILESYSTEM MOUNTPOIT ==================================================== data /data data/u01 /u01 rpool /rpool rpool/ROOT legacy rpool/ROOT/solaris /a rpool/dump - rpool/export /export rpool/export/home /export/home rpool/savecore /var/crash/solaris rpool/swap -
Performing Initial Configuration of Oracle Exadata Database Machine
The following procedure describes how to perform the initial configuration of Oracle Exadata Database Machine. You will need the configuration files generated by Oracle Exadata Database Machine Configurator Spreadsheet.
-
Open a console session to the first database server. The first database server is the lowest database server in the rack, which is rack position U16.
Note:
The network connection and configuration for Oracle Exadata Database Machine has not occurred yet.See Also:
Appendix E, "Cabling Tables for Oracle Exadata Database Machine" for rack layout -
Log in as the
root
user on the first database server.Note:
If you do not have the password for theroot
user, then contact Oracle Support Services. -
Copy the configuration files generated by Oracle Exadata Database Machine Configurator Spreadsheet to the database server using one of the following methods:
-
Using a USB flash drive on Linux:
-
Copy the configuration files to a USB flash drive.
-
Connect the USB drive to the database server.
-
Locate the USB drive using the following command:
for x in `ls -1 /sys/block`; do udevinfo -a -p /sys/block/$x | grep \ -iq 'DRIVER=="usb-storage"'; if [ $? -eq 0 ]; then echo /dev/${x}1; \ fi ; done
-
Create a directory on the database server using the following command:
# mkdir /mnt/usb
-
Mount the device. Use the device name given in step c. The following is an example of the command.
# mount -t vfat /dev/sdb1 /mnt/usb
-
Copy the configuration files from the USB flash drive to the
onecommand
directory on the database server using the following command:# cp /mnt/usb/path_to_files/*.* /opt/oracle.SupportTools/onecommand
In the preceding command, path_to_files is the directory path to the configuration files on the USB flash drive.
-
Unmount the USB flash drive using the following command:
# umount /mnt/usb
-
Remove the USB flash drive from the database server.
-
-
Using a USB flash drive on Oracle Solaris:
-
Copy the configuration files to a USB flash drive.
-
Connect the USB drive to the database server.
-
Check if the USB flash drive has been automatically mounted to
/media/
using a command similar to the following:mount | grep -i usb
The output should be similar to the following:
/media/USB2FlashStorage on /dev/dsk/c5t0d0p0:1 read/write/nosetuid/nodevices/rstchown/hidden/nofoldcase/clamptime/ \ noatime/timezone=28800/dev=981190 on Mon Apr 4 12:47:39 2011
If the USB flash drive is not mounted automatically, then mount it manually.
-
Identify the device for the USB flash drive using the following command:
# mformat
The output should be similar to the following. In this example, the device name is
c5t0d0
.Looking for devices... 1. Logical Node: /dev/rdsk/c5t0d0p0 Physical Node: /pci@0,0/pci108e,4844@1d,7/hub@3/storage@2/disk@0,0 Connected Device: Ut163 USB2FlashStorage 0.00 Device Type: Removable Bus: USB Size: 1.9 GB Label: <Unknown> Access permissions: Medium is not write protected.
-
Create a directory on the database server using the following command:
# mkdir /media/USB2FlashsStorage
-
Mount the device using the device name identified in step d. The following is an example of the command:
# mount -F pcfs /dev/dsk/c5t0d0p0:c /media/USB2FlashStorage
-
Copy the configuration files from the USB flash drive to the
onecommand
directory on the database server using the following command:# cp /media/USB2FlashStorage/path_to_files/*.* /opt/oracle.SupportTools/onecommand
In the preceding command, path_to_files is the directory path to the configuration files on the USB flash drive.
-
Unmount the USB flash drive using the following command:
# umount /media/USB2FlashStorage
-
Remove the USB flash drive from the database server.
-
-
Using a temporary network connection:
This option is only needed if there were IP address conflicts, and Oracle Exadata Database Machine is not already connected to the network.
-
Connect a network cable to the Ethernet port labeled NET0 on the back of the first database server.
-
Bring up the eth0:1 interface as follows:
For Linux:
# ifconfig eth0:1 real_ip netmask real_netmask up
In the preceding command, real_ip is the IP address to assign to the database server, and real_netmask is the subnet mask to assign to the database server.
For Oracle Solaris:
ifconfig eth0:1 plumb ifconfig eth0:1 real_ip netmask real_netmask up
-
Add the default route as follows:
For Linux:
# route add -net 0.0.0.0 netmask 0.0.0.0 gw real_gateway
In the preceding command, real_gateway is the default gateway to use for the database server.
For Oracle Solaris:
route add default real_gateway
-
Copy the configuration files to the
/opt/oracle.SupportTools/onecommand
directory on the database server using a file transfer utility such as scp. -
Disconnect the network cable from Ethernet port NET0.
Note:
It is not necessary to reconfigure the network because the network will be reconfigured automatically during the restart step.
-
-
-
Determine the Oracle Exadata Storage Server Software release currently installed on Oracle Exadata Database Machine using the following command:
# imagehistory
The choice of configuration file depends on the version and imaging mode shown in the output from the command. For Oracle Exadata Database Machine at release 11.2.2.1.0 or later, use the
preconf.csv
file. For earlier releases, do the following:-
If the version is
11.2.1.3.0
and the imaging mode isfresh
as shown in the following, then use thepreconf-11-2-1-2-2.csv
file.Version : 11.2.1.3.0 Image activation date : 2010-08-13 13:12:59 -0700 Imaging mode : fresh Imaging status : success
-
If the imaging mode is
patch
as shown in the following, then use thepreconf-11-2-1-1-0.csv
file.Version : 11.2.1.3.0 Image activation date : 2010-08-21 18:27:54 -0500 Imaging mode : patch Imaging status : success
If the release is release 11.2.1.3.1 or later, then go to step 5. If the release is earlier than release 11.2.1.3.0, then do the following:
-
Get the network interface MAC addresses for all nodes by running the
fetch.macs.sh
script. Send the output to grep for the required customer network interfaces and redirect to a file. MAC addresses for eth1 and eth2 are not needed if BONDETH0 is used. When using the script, usefull
,half
, orquarter
to reflect the type of rack.Note:
As of release 11.2.2.1.0, the names used for bonding changed. The names are BONDIB0 for the InfiniBand bonding and BONDETH0 for Ethernet bonding. In earlier releases, the names were BOND0 and BOND1, respectively.The following is an output example for Exadata Database Machine Full Rack when eth0, eth1 and eth3 are used, and BONDETH0 is not used:
#cd /opt/oracle.SupportTools/firstconf #fetch_macs.sh full |grep 'eth0\|eth1\|eth3' > mymacs #cat mymacs 192.168.1.22: fcell02 eth0 00:14:4F:CA:F8:E6 192.168.1.21: fcell01 eth0 00:21:28:47:3D:7E ….. 192.168.1.9: fdb02 eth0 00:21:28:47:3D:7E 192.168.1.8: fdb01 eth0 00:14:4F:CA:FA:AE ….. 192.168.1.9: fdb02 eth1 00:11:1F:CA:CC:2F 192.168.1.8: fdb01 eth1 00:14:4F:E7:89:89 ….. 192.168.1.9: fdb02 eth3 00:11:1F:CA:CF:2G 192.168.1.8: fdb01 eth3 00:14:4F:E8:89:89
The following is an output example for Exadata Database Machine Full Rack when using BONDETH0, eth0, and eth3 are used:
#cd /opt/oracle.SupportTools/firstconf #fetch_macs.sh full |grep 'eth0\|eth3' > mymacs #cat mymacs example of output: 192.168.1.22: fcell02 eth0 00:14:4F:CA:F8:E6 192.168.1.21: fcell01 eth0 00:21:28:47:3D:7E …. 192.168.1.9: fdb02 eth0 00:21:28:47:3D:7E 192.168.1.8: fdb01 eth0 00:14:4F:CA:FA:AE 192.168.1.9: fdb02 eth3 00:11:1F:CA:CC:2F 192.168.1.8: fdb01 eth3 00:14:4F:E7:89:89
-
Use the vi editor to update the
preconf-11-2-1-1-0.csv
file with the network interface MAC address values in themymacs
file. Make sure to match the node type and number, and avoid typing errors. Network interface eth0 always needs to be updated on the database servers and Exadata Storage Servers. The network MAC addresses needs to be updated for any customer-specified unbonded network interface such as eth1, eth2, and eth3 on the database servers.
-
-
Validate the
preconf.csv
file using the following command:# /opt/oracle.cellos/ipconf.pl -verify \ -preconf /opt/oracle.SupportTools/onecommand/preconf.csv
In the preceding command, preconf.csv is the name of the configuration file. The name is
preconf-11-2-2-1-0.csv
for releases 11.2.2.1 and later,preconf-11-2-1-2-2.csv
for release 11.2.1.3.0, andpreconf-11-2-1-1-0.csv
for releases earlier than release 11.2.1.3.0.Note:
All validation errors must be fixed before continuing. -
Run the
applyconfig.sh
script using the following command:# cd /opt/oracle.SupportTools/firstconf # ./applyconfig.sh -preconf /opt/oracle.SupportTools/onecommand/preconf.csv \ -dbm-node-ips size
In the preceding command, size is the size of Oracle Exadata Database Machine. Use
full
for Exadata Database Machine Full Rack,half
for Exadata Database Machine Half Rack, orquarter
for Exadata Database Machine Quarter Rack.The
applyconfig.sh
script performs network configuration for all database servers and Exadata Storage Servers. All servers restart during the process. -
Connect the corporate network cable for the management network to the Cisco Ethernet switch.
-
Connect the corporate network cables for the client access network to the database servers.
-
Restart all database servers and Exadata Storage Servers.
See Also:
"Powering On and Off Oracle Exadata Rack" for information about restarting the servers -
Log in to the first database server to verify network connectivity using the following commands:
# /opt/oracle.SupportTools/onecommand #./checkip.sh -m post_applyconfig
Note:
The preceding command can be run on a Linux server outside of Oracle Exadata Database Machine to verify network connectivity to Oracle Exadata Database Machine. Copy thecheckip.sh
anddbm.dat
files to the external system, and run the command as shown in this step. -
Use the Oracle OneCommand utility to install and configure the software for Oracle Exadata Database Machine.
Loading the Configuration Information and Installing the Software
The Oracle OneCommand utility loads the network settings, creates the user accounts, installs Oracle Database software, and secures the system based on the information the configuration files. You can also choose to do the process manually. The following sections describe each process.
-
Configuring Oracle Exadata Database Machine Using Oracle OneCommand Utility
-
Configuring Oracle Database and Oracle ASM Instances for Oracle Exadata Database Machine Manually
See Also:
Table 4-1, "Files for Oracle Exadata Database Machine Configuration" for descriptions of the configuration files generated by Oracle Exadata Database Machine Configurator SpreadsheetConfiguring Oracle Exadata Database Machine Using Oracle OneCommand Utility
The Oracle OneCommand utility is used to configure Oracle Exadata Database Machine. The utility works with the information provided in Oracle Exadata Database Machine Configurator Spreadsheet.
The following procedure describes how to configure Oracle Exadata Database Machine using the Oracle OneCommand utility:
-
Log in as the
root
user on the first database server.Note:
If you do not have the password for theroot
user, then contact Oracle Support Services. -
Go to My Oracle Support at
-
Download the latest Oracle OneCommand utility, and follow the steps in the associated README file. See My Oracle Support note 888828.1 for details.
-
Download all necessary Oracle Exadata Storage Server Software and Oracle Database patches. See My Oracle Support note 888828.1 for details.
-
Apply the Oracle OneCommand utility update according to the instructions included in the
README
file. -
Change to the following directory:
# cd /opt/oracle.SupportTools/onecommand
-
Run the appropriate command for your release as follows. Oracle recommends using the
-s
option to run the script one step at a time.-
For Oracle Database 11g Release 2 (11.2.0.2)
./deploy112.sh -l
-
For Oracle Database 11g Release 2 (11.2.0.3)
./deploy11203.sh -l
In the preceding command,
-l
is the option to list the configuration steps. The following table lists the options for the script.Option Description -h
Describe options for the script. -s
n
Run step n of the Oracle OneCommand utility. -r
n-N
Run steps n through N of the Oracle OneCommand utility. -l
List the steps of the Oracle OneCommand utility.
-
Table 6-41 lists the steps of the Oracle OneCommand utility. The steps before grid disk creation (step 17) can be run more than once. Once the script is run to the end, it cannot be run again.
Table 6-41 Steps of the Oracle OneCommand Utility
Step Number | Name | Description |
---|---|---|
Step 0 | ValidateEnv | Validate the environment. |
Step 1 | CreateWorkDir | Create work directory. |
Step 2 | UnzipFiles | Unzip files. |
Step 3 | setupSSHroot | Setup SSH for the |
Step 4 | UpdateEtcHosts | Update the |
Step 5 | CreateCellipinitora | Create the |
Step 6 | ValidateIB | Validate the InfiniBand network. |
Step 7 | ValidateCell | Validate the cells. |
Step 8 | PingRdsCheck | Check RDS using the |
Step 9 | RunCalibrate | Calibrate the system. |
Step 10 | CreateUsers | Create users. |
Step 11 | SetupSSHusers | Set up SSH for the user accounts. |
Step 12 | CreateGridDisks | Create the grid disks. |
Step 13 | GridSwInstall | Install the grid software. |
Step 14 | PatchGridHome | Patch the grid home software. |
Step 15 | RelinkRDSGI | Relink Reliable Data Socket (RDS) for the Grid Infrastructure. |
Step 16 | GridRootScripts | Run the grid root scripts. |
Step 17 | DbSwInstall | Install the Oracle Database software. |
Step 18 | PatchDBHomes | Patch the database software. |
Step 19 | CreateASMDiskgroups | Create the Oracle ASM disk groups. |
Step 20 | DbcaDB | Run Oracle Database Configuration Assistant. |
Step 21 | DoUnlock | Unlock Oracle Grid Infrastructure. |
Step 22 | RelinkRDSDb | Relink RDS protocol. |
Step 23 | LockUpGI | Lock Oracle Grid Infrastructure. |
Step 24 | SetupCellEmailAlerts | Set up e-mail alerts for Exadata Storage Servers. |
Step 25 | ApplySecurityFixes | Apply any security fixes. |
Step 26 | ResecureMachine | Secure Oracle Exadata Database Machine. |
To configure a database instance to access cell storage, ensure that the COMPATIBLE parameter is set to 11.2.0.2
in the database initialization file.
You can view the initialization parameter file from the operating system or use the SQL*Plus administration command SHOW PARAMETER
if the database is running. For example:
SQL> SHOW PARAMETER SQL> SHOW PARAMETER compatible
If necessary, set the COMPATIBLE initialization parameter in the initialization parameter file to 11.2.0.2
or later. For example:
COMPATIBLE='11.2.0.2'
The COMPATIBLE parameter cannot be changed dynamically. If you change the value in the initialization parameter file, then the database must be shut down and restarted for the change to take effect.
Configuring Initialization Parameters for an Oracle ASM Instance
To enable an Oracle ASM instance to discover and access Exadata Storage Server grid disks, you must configure the ASM_DISKSTRING initialization parameter as follows:
-
Set the ASM_DISKSTRING initialization parameter to the empty string (
''
) to discover all grid disks on the cells listed in thecellip.ora
file. This causes Oracle ASM discovery for/dev/raw/*
and any ASMLIB disks on any non-Exadata Storage Servers. -
Set the ASM_DISKSTRING initialization parameter to restrict the discovery by Oracle ASM to only specific sets of disks, as shown in the examples.
-
To discover only a subset of cells, use a pattern for the ASM_DISKSTRING value such as the following:
o/cell-connect-info-pattern/griddisk-name-pattern.
For example:
ASM_DISKSTRING = 'o/*/data*'
-
To discover non-Exadata Storage Server disks and all grid disks on all the cells specified in the
cellip.ora
file, use a wildcard pattern for Exadata Storage Server grid disks. For example:ASM_DISKSTRING = '/dev/rdsk/disk*', 'o/*/*'
-
You can change the ASM_DISKSTRING initialization parameter when the Oracle ASM instance is running with the SQL ALTER SYSTEM
command. If you edit the ASM_DISKSTRING initialization parameter in the initialization parameter file when the Oracle ASM instance is running, then the Oracle ASM instance must be shut down and restarted for the change to take effect.
Installing Oracle Enterprise Manager Grid Control on Oracle Exadata Database Machine
The following procedure describes how to install Oracle Enterprise Manager Grid Control on Oracle Exadata Database Machine:
-
Create the
/u01/usb
directory on a database server. -
Copy the USB kit to the
/u01/usb
directory. -
Change to the
/tmp
directory. -
Run the following command:
zcat /u01/usb/dbm_context_linux_64.tar.Z | tar xBvpf -
-
Change to the
/scripts
directory using the following command:cd usb/scripts
-
Run the setup script using the following command:
./setup.sh -em | tee /tmp/setup.log
-
Enter the
root
password when prompted by the system. -
Enter
y
orn
when prompted for the Oracle Database home. If the response isn
, then the system will prompt for the correct Oracle Database home. -
Enter the Oracle Enterprise Manager Grid Control user password for each database server when prompted.
-
Enter
y
orn
when prompted to harden the Oracle Database and Oracle ASM passwords. -
Enter
y
orn
when prompted for the Oracle Database home. If the response isn
, then the system will prompt for the correct Oracle Database home.
The script will discover the targets automatically. It may be necessary to manually configure the Oracle Database, Oracle ASM, and Oracle Enterprise Manager Grid Control targets in OMS.
Installing Oracle Enterprise Manager Cloud Control on a Standalone Server to Monitor Oracle Exadata Database Machine
The following procedure describes how to prepare the standalone server to monitor Oracle Exadata Database Machine:
-
Log in as the
root
user on the standalone server. -
Download the latest Oracle Enterprise Manager Cloud Control patches, and follow the steps in the associated README file. See My Oracle Support note 888828.1 for details.
-
Download the latest patch available from Oracle bug database. The bug number is 11852869. The following files must be downloaded:
-
p11852869_111010_Linux-x86-64_1of4.zip
-
p11852869_111010_Linux-x86-64_2of4.zip
-
p11852869_111010_Linux-x86-64_3of4.zip
-
p11852869_111010_Linux-x86-64_4of4.zip
-
-
Copy the downloaded files to the
/tmp/emkit
directory. -
Change to the
/tmp/emkit
directory. -
Run commands similar to the following to unzip the files:
unzip p12960610_121010_Linux-x86-64_1of4.zip unzip p12960610_121010_Linux-x86-64_2of4.zip unzip p12960610_121010_Linux-x86-64_3of4.zip unzip p12960610_121010_Linux-x86-64_4of4.zip
-
Run the following command:
cat CloudControl_Standalone00 CloudControl_Standalone01 \ CloudControl_Standalone02 CloudControl_Standalone03 > \ CloudControl_Standalone_12.1.0.1.0_Linux_x86.tar.Z
-
Run the following command as the
root
user:tar xBvpf CloudControl_Standalone_12.1.0.1.0_Linux_x86.tar.Z
-
Change to the
/tmp/emkit/common
directory. -
Run the following command as the
root
user:(sh setupem.sh -emOnly -emUser em_user -emBase em_base|tee /tmp/setupem.log) \ 3>&1 1>&2 2>&3 | tee /tmp/setupem.err
In the preceding command, em_user is the user who owns Oracle Management Server (OMS), and em_base is the directory to be used when creating OMS and agent home. The operating system user must exist and belong to the oinstall and dba groups.
Note:
If the messageAmbiguous output redirect
appears after running the command, then change to the bash shell. -
Log in to OMS as the
sysman
user using the OMS URL, and verify the setup as follows:-
Accept the license agreement.
-
Verify the agent and host targets appear under the Targets tab.
-
-
Discover the Management Repository instance as follows:
-
Log in to Oracle Enterprise Manager Cloud Control as the
sysman
user. -
Select Databases from the Targets tab.
-
Select Search List.
-
Click Add.
-
Click the search icon.
-
Select EM host.
-
Click Continue.
-
Click the configuration icon when the database is discovered.
-
Enter the monitor password on the Configure Database Instance page.
-
Click Test Connection.
-
Use the wizard
-
Save the targets.
-
-
Enable Oracle Database Lifecycle Management Pack as follows:
-
Select Management Packs from the Setup page.
-
Select Management Pack Access.
-
Select Pack Access Agreed.
-
Click Apply.
-
Log out, and then log back in.
-
-
Change the passwords for the database and OMS to secure the system.
Installing Oracle Enterprise Manager Cloud Control Agents on Oracle Exadata Database Machine
The following procedure describes how to install Oracle Enterprise Manager Cloud Control agents on Oracle Exadata Database Machine.
Note:
The steps in this procedure assume that Oracle Enterprise Manager Cloud Control information was entered into the Oracle Exadata Database Machine Configurator Spreadsheet, and the/opt/oracle.SupportTools/onecommand/em.param
file is available. If not, then create the file. Refer to
"About the em.param File" for an example of the file.
-
Log in as the
root
user on the first database server. -
Create the
/tmp/emkit
directory on the first database server on Oracle Exadata Database Machine. -
Download the Oracle Enterprise Manager Cloud Control Agent kit from Oracle, selecting the kit that is appropriate for your system.
-
Copy the downloaded kit to the
/tmp/emkit
directory. -
Change to the
/tmp/emkit
directory. -
Run the following commands to unzip the file:
-
For Linux:
unzip p12960596_121010_Linux-x86-64.zip tar xBvpf CloudControl_Agent_12.1.0.1.0_LINUX_X64.tar.z
-
For Oracle Solaris:
unzip p12960596_121010_Solaris86-64.zip tar xBvpf CloudControl_Agent_12.1.0.1.0_SOLARIS_X64.tar.z
-
-
Change to the
/tmp.emkit/common
directory. -
Run the setup script using the following command:
(sh setupem.sh | tee /tmp/setupag.log) 3>&1 1>&2 2>&3 | tee /tmp/setupag.err
Setup for SSH for the
root
user will start on all servers. -
Enter the password for each server when prompted. The system prompts twice for each password.
-
Enter the Management agent registration password for the agent on the local server.
-
Discover Oracle Exadata Database Machine as follows:
-
Log in to Oracle Enterprise Manager Cloud Control.
-
Select Add Non-Host Targets Using Guided Process.
-
Select Oracle Exadata Database Machine as the target type.
-
Click Add Using Guided Discovery.
-
Use the wizard.
-
-
Discover the grid infrastructure as follows:
-
Select Add Targets from the Setup page.
-
Select Add Targets Manually.
-
Select Add Non-Host Targets Using Guided Process.
-
Select Oracle Exadata Database Machine as the target type.
-
Click Add Using Guided Discovery.
-
Use the wizard.
-
-
Discover Oracle RAC as follows:
-
Select Add Targets from the Setup page.
-
Select Add Targets Manually.
-
Select Add Non-Host Targets Using Guided Process.
-
Select Oracle Database, Listener and Automatic Storage Management as the target type.
-
Click Add Using Guided Discovery.
-
Use the wizard.
-
Configure the Oracle RAC databases. The dbsnmp password is required for this step.
-
Configure Oracle ASM. The asmsnmp password is required for this step.
-
-
Enter the
em_user
user password for the server when prompted by the system. -
Enter
y
orn
when prompted for the Oracle Database home. If the response isn
, then the system prompts for the correct Oracle Database home. -
Respond to the prompts, as needed.
Note:
You can configure Oracle Configuration Manager (OCM) using the emocmrsp utility. The utility prompts for My Oracle Support credentials and connection information, and configures OCM collector on all Exadata Storage Servers.The utility creates the
/opt/oracle.SupportTools/dbmcprov/ccr/exadata_homes.csv
file which includes all servers and their Oracle homes. This file can be used with the Mass Deployment utility to re-configure OCM collector.Use the software CSI assigned to Oracle Exadata Database Machine when configuring OCM collector.
-
Remove the SSH setup when prompted by the system.
See Also:
My Oracle Support note 1110675.1 for information about configuring plug-insThe /opt/oracle.SupportTools/onecommand/em.param
file is generated by Oracle Exadata Database Machine Configurator Spreadsheet. The following is an example of the file:
# This is em.param file # Written : 3/30/2011 EM_VERSION=1.0 OMS_LOCATION=REMOTE_CORP_GC EM_BASE=/myem/myemhome OMS_HOST=myoms.example.com OMS_PORT=4900 EM_CELLS=(cell_node1 cell_node2 cell_node3) EM_COMPUTE_ILOM_NAME=(dbhost1-ilom dbhost2-ilom) EM_COMPUTE_ILOM_IP=(x.x.x.x x.x.x.x) machinemodel="X2-2 Quarter rack" EM_USER=oracle EM_PASSWORD=******* swikvmname=my-kvm swikvmip=x.x.x.x swiipname=mycisco swiipip=x.x.x.x swiib1name=my-ib1 swiib1ip=x.x.x.x swiib2name=my-ib2 swiib2ip=x.x.x.x swiib3name=my-ib3 swiib3ip=x.x.x.x pduaname=my-pdua pduaip=x.x.x.x pdubname=my-pdub pdubip=x.x.x.x
In the preceding example, the OMS_LOCATION
setting has two options. The setting OMS_LOCATION=REMOTE_CORP_GC
means that the agent is configured on this Oracle Exadata Rack. The other setting is OMS_LOCATION=NO_GC
, which means the agent will not be configured on Oracle Exadata Rack. The settings are based on the Install 11g Management Agent option in Oracle Exadata Database Machine Configurator Spreadsheet. The OMS_PORT
setting is the upload port.
Note:
If Oracle Enterprise Manager Cloud Control configuration is being run after the initial installation of Oracle Exadata Database Machine, then verify the contents of the em.param file are still valid.The following table lists the em.param switch parameters and their descriptions.
Switch Parameter | Description |
---|---|
swikvmname | Name of the KVM switch. |
swikvmip | IP address of the KVM switch. |
swiipname | Name of the Cisco switch. |
swiipip | IP address of the Cisco switch. |
swiib1name | Name of first InfiniBand switch. This is a leaf switch. |
swiib1ip | IP address of first InfiniBand switch. |
swiib2name | Name of second InfiniBand switch. This is a leaf switch. |
swiib2ip | IP address of second InfiniBand switch. |
swiib3name | Name of third InfiniBand switch. This is a spine switch. |
swiib3ip | IP address of third InfiniBand switch. |
pduaname | Name of PDU-A. |
pduaip | IP address of PDU-A. |
pdubname | Name of PDU-B. |
pdubip | IP address of PDU-B. |
Removing Oracle Enterprise Manager Grid Control Agents from Oracle Exadata Database Machine
The following procedure describes how to manually remove Oracle Enterprise Manager Grid Control agents from Oracle Exadata Database Machine:
-
Follow the deinstallation instructions in Oracle Enterprise Manager Grid Control Advanced Installation and Configuration Guide.
-
Remove the kit from the
/tmp
directory using the following command:$ rm -rf /tmp/usb
-
Remove
dbmcprov
from the/opt/Oracle.SupportTools
directory using the following command:$ cd /opt/Oracle.SupportTools/ $ rm -rf dbmcprov
-
Remove the targets as follows:
-
Oracle Enterprise Manager Grid Control was installed on Oracle Exadata Database Machine:
-
Log in as the
root
user on the first database server. -
Change to the
/opt/Oracle.SupportTools/dbmcprov/scripts
directory. -
Run the following command:
./rollback.sh -cleanAgents
-
Enter the root password when prompted by the system.
-
-
Oracle Enterprise Manager Grid Control was installed on standalone server:
-
Log in as the
root
user on the standalone server. -
Change to the
/opt/Oracle.SupportTools/dbmcprov/scripts
directory. -
Run the following command:
./rollback.sh -cleanStandAloneOMS
-
-
-
Delete the targets from OMS. The following targets should be deleted:
-
Oracle ASM targets for Oracle Exadata Database Machine
-
Oracle Cluster database targets
-
Oracle Database instance targets
-
Cisco switch targets
-
Oracle InfiniBand switch targets
-
KVM switch targets
-
PDU switch targets
-
Exadata Storage Server targets
-
Oracle ILOM server targets
-
Oracle Database server targets
-
Oracle Database agent targets
-
Listeners on Oracle Database server targets
-
Removing Oracle Enterprise Manager Cloud Control Agents from Oracle Exadata Database Machine When Using a Standalone Server
The following procedure describes how to remove Oracle Enterprise Manager Cloud Control agents from Oracle Exadata Database Machine:
-
Log in as the
root
user on the first database server. This is the same server that the installation was done on. -
Change to the
/opt/Oracle.SupportTools/emkit/common
directory. -
Run the following command:
(sh rollback.sh -cleanAgents -emBase em_base | tee /tmp/cleanupag.log) \ 3>&1 1>&2 2>&3 | tee /tmp/cleanupag.err
In the preceding command, em_base is the directory for the agent.
-
Remove the Oracle Exadata Database Machine targets from the Oracle Enterprise Manager Cloud Control server. The following targets should be deleted:
-
Oracle ASM targets for Oracle Exadata Database Machine
-
Oracle Cluster database targets
-
Oracle Database instance targets
-
Cisco switch targets
-
Oracle InfiniBand switch targets
-
KVM switch targets
-
PDU switch targets
-
Exadata Storage Server targets
-
Oracle ILOM server targets
-
Listeners on Oracle Database server targets
-
Host targets
-
Oracle agent targets
-
Oracle home targets
-
Removing Oracle Enterprise Manager Cloud Control from a Standalone Server
The following procedure describes how to remove Oracle Enterprise Manager Cloud Control from a standalone server:
-
Log in as the
root
user to the first database server where the setup was done. -
Change to the
/opt/Oracle.SupportTools/emkit/common
directory. -
Run the following command:
(sh rollback.sh -cleanStandAloneOMS -emBase em_base | tee /tmp/cleanupem.log) \ 3>&1 1>&2 2>&3 | tee /tmp/cleanupem.err
In the preceding command, em_base is the directory for OMS and agent home.
-
Ensure the directory for OMS and agent home directory, em_base, is empty.
Installing Oracle Enterprise Manager Grid Control on Oracle Exadata Database Machine
The following procedure describes how to install Oracle Enterprise Manager Grid Control on Oracle Exadata Database Machine:
-
Create the
/u01/usb
directory on a database server. -
Copy the USB kit to the
/u01/usb
directory. -
Change to the
/tmp
directory. -
Run the following command:
zcat /u01/usb/dbm_context_linux_64.tar.Z | tar xBvpf -
-
Change to the
/scripts
directory using the following command:cd usb/scripts
-
Run the setup script using the following command:
./setup.sh -em | tee /tmp/setup.log
-
Enter the
root
password when prompted by the system. -
Enter
y
orn
when prompted for the Oracle Database home. If the response isn
, then the system will prompt for the correct Oracle Database home. -
Enter the Oracle Enterprise Manager Grid Control user password for each database server when prompted.
-
Enter
y
orn
when prompted to harden the Oracle Database and Oracle ASM passwords. -
Enter
y
orn
when prompted for the Oracle Database home. If the response isn
, then the system will prompt for the correct Oracle Database home.
The script will discover the targets automatically. It may be necessary to manually configure the Oracle Database, Oracle ASM, and Oracle Enterprise Manager Grid Control targets in OMS.
总结:一体机内部有大量的写好的脚步,从上文档发现很多配置都是脚步!