Print-friendly Version

Scripts for Automating Network Systems Checks

Victor Feng, April 2008

Introduction

In my previous article, Scripts for Automating System Checks, I described how to check logs, check for new patches, and check for disk errors automatically on a single system using the checkmessages, checkpatch, and checkdisk scripts. If those scripts find anything wrong, the system sends email to notify the system administrator.

This tech tip provides five scripts for automating the same system checks on networked systems that run the Solaris 9 or 10 Operating System. If these scripts find anything wrong, email is sent to the system administrator.

In order to send out email, sendmail has to be running. By default, sendmail runs on a system, but in some situations, sendmail is disabled, for instance, when another email system (such as Sun Java System Messaging Server) is using port 25. To handle such situations, you need to copy the necessary information for the systems in question to a shared place, and process the information using a system that has sendmail running.

For the purposes of this tech tip, assume that host02 does not have sendmail, and it has an NFS shared folder, /report, that is writable by both host01 and host03. Also assume that host03 does not have sendmail, and that host01 has sendmail and is going to process the information for both host02 and host03. If there is anything wrong with either host02 or host03, host01 will send email to notify the system administrator.

Source Code for the Scripts

Here are the five scripts this tech tip provides:

• The checkmessages_net script checks the logs on the hosts. If anything wrong is found, the script sends email to the system administrator. This script should be scheduled to run daily using cron.

  Here's the source code for the checkmessages_net script. Please rename it so it has a .sh extension instead of a .txt extension.

• The checkdisk_net, host02_checkdisk, and host03_checkdisk scripts check predictive disk failure on the hosts. They should be run daily also.

  Here's the source code for the checkdisk_net script. Please rename it so it has a .sh extension instead of a .txt extension.

  Here's the source code for the host02_checkdisk script. Please rename it so it has a .sh extension instead of a .txt extension, and then install it on the machine that you want to be host02 before you run the checkdisk_net or checkmessages_net script.

  Here's the source code for the host03_checkdisk script. Please rename it so it has a .sh extension instead of a .txt extension, and then install it on the machine that you want to be host03 before you run the checkdisk_net or checkmessages_net script.

• The checkpatch script checks for newly available patches for a system. It, too, should be run daily.

  Here's the source code for the checkpatch script. Please rename it so it has a .sh extension instead of a .txt extension.

The checkmessages_net Script

The checkmessages_net script on host01 first processes its own /var/adm/messages file. Then it processes the messages file of host02.

Before you run the checkmessages_net script, you need to copy the log file of host02 from /var/adm/messages to the shared location, /report. Use a cron job to do this on NFS server host02, as follows:

58 23 * * * /usr/bin/cp /var/adm/messages /report/host02_messages

Then, on host01, you can use a cron job to run checkmessages_net a little later:

59 23 * * * /home/<username>/bin/checkmessages

Do something similar on host03 to copy its log file before running the checkmessages_net script:

58 23 * * * /usr/bin/cp /var/adm/messages
/net/host02/report/host03_messages

Also, before running checkmessages_net, assign your email address to the variable MAIL_RECEIVER.

The checkdisk_net Script

When host02 runs the host02_checkdisk script, the host02_checkdisk script creates a new file, host02_diskerror.txt, in the shared folder, /report, if a disk error is found.

Then the checkdisk_net script on host01 sends out email based on the existence of the host02_diskerror.txt file, and then it deletes the file.

Therefore, the host02_checkdisk script should be run before the checkdisk_net script. For example, on host02, you could use the following cron command:

30 23 * * * /home/<username>/bin/host02_checkdisk

On host01, you could use this cron command:

31 23 * * * /home/<username>/bin/checkdisk

Similarly, for host03, the host03_checkdisk script needs to be scheduled to run before the checkdisk_net script is run on host01:

On host03, you could use this cron command:

30 23 * * * /home/<username>/bin/host03_checkdisk

In a similar way to checking for disk errors, you can check for newly available patches for host02 and host03, as described next.

The checkpatch Script

The checkpatch script uses the Patch Check Advanced (PCA) tool, which you can download for free.

Some new patches might be not related to your system. If you do not want to be notified about them again, you can add them to the script using lines similar to the following:

IGNORE="$IGNORE --ignore 112925"
IGNORE="$IGNORE --ignore 112960"

After you patch your systems, you should remove such lines from the script so that you are notified if new, related patches become available.

The checkpatch script checks security patches. But you can modify the script to check for all available patches by changing missings to missing.

Again, you can use cron to run the script daily.

A Sun Online account is now required to download patchdiag.xref. Therefore, before running the checkpatch script, assign the user name and password for your Sun Online account to the USERNAME and PASSWORD variables in the checkpatch script.

Synchronizing the Time Clock of the Servers

As you can see, the time clock among the servers needs to be synchronized. You can do this through the Network Time Protocol (NTP). If you are not familiar with NTP, here is a fast way to set it up.

1. Pick one server as the NTP server, for example, host01, and do the following:

Note: It is best to choose the server on which sendmail is running.

# vi /etc/inet/ntp.conf
  server 127.127.1.0

Or, you can replace 127.127.1.0 with your authoritative DNS or Microsoft Active Directory server.

2. On the other servers, for example, host02 and host03, do the following:

# vi /etc/inet/ntp.conf
  Server host01

3. Start the NTP service on each server, starting the service on the NTP server first:

On the Solaris 9 OS:

# /etc/init.d/xntpd start

On the Solaris 10 OS:

# svcadm enable ntp

From now on, host02 and host03 will have the same time as host01.

Unless otherwise licensed, code in all technical manuals herein (including articles, FAQs, samples) is provided under this License.