1.1 Quick Start Guide
This guide walks you through the basic steps to install and configure ORAchk and EXAchk. Review other topics in this guide for additional information.
1.1.1 Overview of ORAchk and EXAchk
ORAchk and EXAchk provide a lightweight and non-intrusive health check framework for the Oracle stack of software and hardware components. If you have an engineered system other than Oracle Database Appliance, use EXAchk. For all other systems, use ORAchk.
ORAchk and EXAchk are provided as a value add-on to your existing support contract. There is no additional fee or license required to run ORAchk and EXAchk.
Features of ORAchk and EXAchk
-
Automated risk identification and proactive notification before business is impacted.
-
Health checks based on critical, reoccurring problems across Oracle customer base.
-
Runs in your environment with no need to send anything to Oracle.
-
Schedule email health check reports.
-
Findings can be integrated into other tools of choice.
1.1.2 Prerequisites
Review the checklist for Bash requirements, SSH connectivity, and required user privileges to run health checks.
1.1.2.1 Bash Requirements
ORAchk and EXAchk are written largely in bash. Bash 3.2 or higher is required.
1.1.2.2 SSH Connectivity and Access
Where health checks are run in a clustered database environment, ORAchk and EXAchk run on a single node and remotely run health checks on all other cluster nodes. This involves remote copy operations to and from targets and remotely running commands non-interactively.
If security restrictions do not permit such actions, then sections of the tool do not run correctly. Alternate command run plans must be developed, tested, and modified based upon whichever results are blocked.
Passwordless SSH equivalency is required to be configured from the user ID that runs ORAchk and EXAchk, on the database where launched, to that same user ID on each remote server. If passwordless SSH is not already configured, then ORAchk and EXAchk configure passwordless SSH automatically, either temporarily or permanently depending on your choice.
Remote operations cannot be performed without this configuration in place. If it is not present, then you must make a separate run on each database in the cluster using the -localonly
command line option, and merge the results together.
For more information, see Understanding and Managing Reports & Output.
To configure SSH user equivalence outside the tool if not already configured, see My Oracle Support Note: 372795.1.
1.1.2.3 Handling of root Passwords
Handling of root passwords depends on whether or not the Expect utility is installed.
If the Expect utility is installed, then specify the root passwords when you run the health checks for the first time. The Expect utility stores the passwords and uses the stored password for subsequent sessions. You are asked if the root password is the same for all remote components such as database, switches, and so forth. If you have configured the same root password for all components, then you need to specify the password only once. If the root password is not the same for all components, then you are prompted to validate the root password each time it is required.
If passwords are entered incorrectly or changed in between the time they are entered and the time they are used, you are notified and relevant checks are skipped. Run the health checks after resolving the issues. If any checks are not run, then they are logged in the report output.
See Expect for more information about the Expect utility.
1.1.2.4 Deciding Which User to Run As
Some health checks require root access, so an important prerequisite is deciding the user that you want to use when you run health checks.
-
Run as root (Recommended): In this model, the root user id is used to launch a complete run. The ORAchk/EXAchk process running under the root user id then uses the
su
command to run commands as one of the lower privileged owners of the Oracle Database or grid homes. Lower privileged accounts cannot elevate access to run root level checks. This approach can have advantages in role-separated environments or environments with more restrictive security. -
Run as Oracle Database or grid home owner: In this method, ORAchk/EXAchk is launched from either the Oracle Database or grid home owner user id. The user id that launched the run must have the ability to elevate access to the root user to run the root level checks. This approach may require multiple runs in role separated environments, and more restrictive security requirements may not permit the elevation.
There are a number of options:
-
Skip the checks that require root access.
-
Provide the root user id and password when prompted.
-
Configure SUDO
If using SUDO, then the temporary directory used by ORAchk/EXAchk, which is $HOME, by default must be in the entry in the/etc/sudoers
file corresponding to the user running the health checks:user ALL=(root) NOPASSWD:TEMP_DIR/root_exachk.sh
For example,oracle ALL=(root) NOPASSWD:/root/root_exachk.sh
-
Pre-configure passwordless SSH connectivity.
Note:
Do not use environment variables such as $HOME in the/etc/sudoers
file. If you use, then they are not evaluated. -
1.1.2.5 Tool Specific Prerequisites
Review ORAchk and EXAchk specific prerequisites.
For ORAchk, see ORAchk Scope and Supported Environments.
For EXAchk, see:
1.1.2.6 Data Entry Terminal Considerations
You use any supported Unix and Linux terminal type (character mode terminal, ILOM, VNC server) to launch ORAchk/EXAchk and respond to prompts during interactive run, or configuring the daemon.
Each terminal type has advantages and disadvantages, and the impact of a dropped network connection varies based upon the terminal type used. For example, in an interactive run using a character mode terminal, if all prompts have been answered prior to the network drop, and the update messages are scrolling by, then the run finishes if the network connection is dropped. If the network connection drops before all input prompts are answered, then the running process hangs and have to be manually cleaned up when connectivity is restored. Using a remote connection to a VNC server running on the database where ORAchk/EXAchk is launched can minimize network drop interruptions.
If accessibility software or devices are used which preclude the use of a VNC server, and network drops are experienced, you should work with your network team and system administrator to determine the root cause and adjust the environment as required. For example, if an accessibility aid inserts suspensions and restarts the interactive process running ORAchk/EXAchk that leads to an operating system timeout due to terminal inactivity, then the environment's inactivity timeouts should be lengthened before running the commands.
An assistive tool leading to a timeout at the operating system level due to terminal inactivity is not specific to or a defect in ORAchk/EXAchk, as the timeout could happen to any process being managed by the assistive technology.
1.1.3 Installation
Follow the steps sequentially to deploy ORAchk and EXAchk successfully.
Note:
If your Oracle Exadata Database Machine participates in the Oracle Platinum Services: Exadata Exachk Automation Project, then there is a separate installation method described in Document: 2043991.1
- Download the latest version of the appropriate health check tool zip file.
-
For ORAchk, download
orachk.zip
, ororachk_idm.zip
for ORAchk with IAM support. Both the files are available at Document ID 1268927.2. -
For EXAchk, download
exachk.zip
, that is available at Document ID 1070954.1.
-
- Copy the zip file to the installation directory on the system(s) you want to check.
Note:
ORAchk/EXAchk is Oracle RAC database cluster aware, so you only need to install on one node of the cluster to be able to check all nodes in the cluster. - As the oracle software install user, extract the zip file:
$ unzip orachk.zip
$ unzip exachk.zip
Note:
ORAchk/EXAchk can be staged on a shared network drive if the performance is acceptable enough.
To run ORAchk/EXAchk on read-only NFS server, you must modify the permissions of .cgrep
and scripts inside .cgrep
to at least 555
.
chmod –R 555 .cgrep’
1.1.4 Configuring Daemon Mode
Use the daemon to configure automatic health check runs at scheduled times.
Note:
If you have an Oracle engineered system, then in addition to these usage steps, follow the system specific instructions:
- Set the daemon properties.At a minimum, set AUTORUN_SCHEDULE and NOTIFICATION_EMAIL. For other options and more details, see Automated Daemon Mode Operation.
$ ./orachk –set “AUTORUN_SCHEDULE=3 * * 0 ;NOTIFICATION_EMAIL=some.body@acompany.com”
$ ./exachk –set “AUTORUN_SCHEDULE=3 * * 0 ;NOTIFICATION_EMAIL=some.body@acompany.com”
The above example instructs the tool to run at 3AM every Sunday and to send the results to
some.body@acompany.com
. - Start the daemon.This should be done either as root (recommended), or as the Oracle Database or Oracle Grid Infrastructure home owner, for more details see Decide Which User to Run As.
$ ./orachk –d start
$ ./exachk –d start
- Answer the questions prompted during startup.
For more information about configuring the daemon, see Automated Daemon Mode Operation.
1.1.5 Email Notification and Report Overview
The following sections provide a brief overview about email notifications, sections of the HTML report output, and ways to generate diff report.
1.1.5.1 First Email Notification
After the first health check run by the daemon, all users specified in the NOTIFICATION_EMAIL list receive an email with the report output attached.
Figure 1-1 First Email Notification
Description of "Figure 1-1 First Email Notification"
1.1.5.2 Health Check Report
Health check reports contains health status of each system grouped under different sections.
The HTML report output contains the following:
-
Health score
-
Summary of health check run
-
Table of contents
-
Controls for report features
-
Findings
-
Recommendations
Details of report output are different on each system, and the report is dynamic; therefore, certain sections are displayed only if applicable.
This gives a brief introduction to the health check HTML reports. For more information, see Understanding and Managing Reports & Output.
Report Health Score and Summary
The heading of the report looks similar to the following image:
-
A high level health score based on the number of checks which passed or failed.
-
A summary of the run, showing things such as where and when it was run, which version was used, how long it took, which user it was run as, and so forth.
Figure 1-2 Report Health Score and Summary
Description of "Figure 1-2 Report Health Score and Summary"
Report Table of Contents and Features
The next section in the report after the summary is the table of contents and report features:
-
The table of contents provides links to each of the major sections within the report.
-
The report features allow you to easily toggle which features are shown in the report.
Figure 1-3 Report Table of Contents and Features
Description of "Figure 1-3 Report Table of Contents and Features"
Report Findings
The next section in the report after the table of contents and features are the findings. These are grouped by technology component and show a row for each health check run including:
-
Check status (FAIL, WARNING, INFO or PASS)
-
Type of Check
-
Check Message
-
Where the check was run
-
Link to expand details for further findings and recommendation
Figure 1-4 Report Findings
Description of "Figure 1-4 Report Findings"
Clicking the view details link expands the finding presenting the recommendation details.
-
What to do to solve the problem
-
Where recommendation applies
-
Where problem doesn’t apply
-
Links to relevant documentation or My Oracle Support Knowledge documents
-
Example of data the recommendation is based on
Figure 1-5 View Report Findings
Description of "Figure 1-5 View Report Findings"
Maximum Availability Architecture (MAA) Score Card
After the groupings of findings, find the Maximum Availability Architecture (MAA) Score card that shows installed software versions checked for noncurrent software and incompatible feature usage:
Figure 1-6 Maximum Availability Architecture (MAA) Score Card
Description of "Figure 1-6 Maximum Availability Architecture (MAA) Score Card"
1.1.5.3 Subsequent Email Notification
After the first email notification, subsequent health check runs compare results of the current run against that of the previous allowing you to easily see if something has changed.
The email sent in the notification contains:
-
System Health Score of this run compared to previous
-
Summary of number of checks run and differences between runs
-
Most recent report result as attachment
-
Previous report result as attachment
-
Diff report as attachment, showing difference between this run and last
Figure 1-7 Subsequent Email Notification
Description of "Figure 1-7 Subsequent Email Notification"
1.1.5.4 Diff Report
The diff report attached to the email notification shows a summary of the differences between the most recent report and the one run immediately before, allowing you to quickly see which check results have changed.
A diff report can also be generated manually any time with the –diff <report1> <report2>
syntax. For more information, see Comparing Two Reports.
Figure 1-8 Health Check Baseline Comparison Summary
Description of "Figure 1-8 Health Check Baseline Comparison Summary"
Figure 1-9 Diff Report
Description of "Figure 1-9 Diff Report"
1.1.6 Recommended On Demand Usage
Recommendations to run health checks on demand.
As well as being run in automated mode via the daemon, the health checks can also be run in on demand manual mode by simply running:
$ ./orachk
$ ./exachk
It is recommended to run the health checks in the following on demand scenarios:
-
Pre or post upgrades
-
Machine moves
-
Hardware failure / repair
-
Problem troubleshooting
-
In addition to go-live testing
When running for pre or post-upgrades all databases registered with the Clusterware are automatically detected and list of databases to check is presented.
Pre-upgrade checks should be run during your upgrade planning phase, the tool prompts you which version you are planning to upgrade to:
$ ./orachk –u –o pre
$ ./exachk –u –o pre
After performing your upgrade you should then run the post upgrade checks:
$ ./orachk –u –o post
$ ./exachk –u –o post
For more information about running in on demand mode, see On Demand Mode Operation.
1.1.7 Maintenance
Oracle recommends two approaches for ORAchk/EXAchk maintenance depending on if your environment has an Internet connection or not.
1.1.7.1 Maintaining Health Checks in an Environment with an Internet Connection
When ORAchk/EXAchk is older than 120 days, the tool prompts you on startup to let it automatically download a newer version from My Oracle Support.
The script prompts for your My Oracle Support login details then checks for download and upgrade, if a later version is available.
Downloads can also be manually requested with the –download
option:
$ ./orachk –download
$ ./exachk –download
$ ./exachk –download Enter your my oracle support username:- some.person@acompany.com Enter your my oracle support password:- Started downloading….. exachk.zip successfully downloaded to /opt/oracle.suptools/exachk/exachk_mybox_040116_043027
1.1.7.2 Maintaining Health Checks in an Environment with No Internet Connection
If you do not have a direct connection to My Oracle Support you can download the latest version of ORAchk/EXAchk from a machine that does have an internet connection, transfer it to a shared network staging location, then set the environment variable RAT_UPGRADE_LOC to point to that staging location.
The next time the tool is started, it detects the later version and prompts you to allow it to upgrade.
- Download the appropriate Health Check tool zip file:
-
For ORAchk, download
orachk.zip
. -
For EXAchk, download,
exachk.zip
.
-
- Transfer the zip file to a shared network staging directory.
- From each machine with a version of the tool you wish to upgrade, set the environment variable RAT_UPGRADE_LOC to point to the network staging directory.
$ export RAT_UPGRADE_LOC=PATH_TO_STAGING_DIRECTORY
The next time ORAchk/EXAchk is started, it looks in the directory specified by RAT_UPGRADE_LOC, if this contains an orachk.zip
or exachk.zip
with a later version, it prompts you to allow it to upgrade.
$ ls /opt/oracle.SupportTools/exachk/latest exachk.zip $ export RAT_UPGRADE_LOC=/opt/oracle.SupportTools/exachk/latest $ ./exachk Latest version of exachk (EXACHK VERSION: 12.1.0.2.7_20160401) is available at /opt/oracle.SupportTools/exachk/latest/ Do you want to upgrade to the latest version of exachk? [y/n][y] exachk has been upgraded to EXACHKVERSION:12.1.0.2.7(DEV)_20160401 Running the latest version…
If you have set RAT_UPGRADE_LOC, but do not yet want to upgrade you can still run ORAchk/EXAchk using the –noupgrade
option:
$ ./orachk –noupgrade
$ ./exachk –noupgrade