Performing Centralized Concurrent Backups with AmandaBy Amy Rich There are a number of enterprise tape backup solutions, such as Sun StorEdge
Enterprise Backup Software, VERITAS NetBackup, and LEGATO NetWorker, geared toward
larger corporations. Many sites, especially smaller ones, don't wish to pay
the license fees that some of these solutions command. For those smaller
sites that want something free yet flexible, Amanda, The Advanced
Maryland Automatic Network Disk Archiver, can be an excellent choice. Amanda is a
collection of C programs designed to run automated centralized concurrent
backups using the vendor's dump software ( The Amanda PhilosophyAmanda uses a client/server model in which the server, called the tape
host, is attached directly to a tape drive or tape changer. Generally the
tape host also has one or more large local storage areas, called holding
disks, which are used to temporarily store data sets from remote machines. If
desired, Amanda can be configured to only back up data sets to the local disks
and not to tape. In this configuration, no tape drive is required. Amanda
can also be configured to run without a holding disk, but this configuration
requires that backups are run sequentially instead of in parallel. The
software on the tape host, generally run from The client software is spawned by Each Amanda configuration on the tape host has three important files,
Amanda requires that each tape in a configuration be labeled before use. This simplistic tape management system prevents tapes with valid data sets from being overwritten and also prevents Amanda from using tapes that are not intended for the current configuration. Each run of Amanda requires one or more new tapes since Amanda does not append to old tapes. Based on settings in the Amanda configuration file, tapes expire and can later be reused. Amanda can also be configured to never expire tapes, which is useful for offsite archive sets. Amanda supports using multiple tapes in a single run, but cannot split a single data set across multiple tapes. This also means it does not support dump images larger than a single tape. If space or bandwidth is at a premium and CPU cycles are not, data sets can be compressed on either the client or the server. Amanda can also be configured to use a tape device that provides hardware compression instead. If Amanda is configured to do a mix of full and incremental backups, the software, not the system administrator, schedules the days on which full or incremental backups occur based upon an internal algorithm. The algorithm takes into account the number of days since the last full backup, the amount of space on the tape, and the number of days until the last full backup will be overwritten. Amanda can also be configured to perform only full backups, if incrementals are not desired. Installing AmandaPreconfiguration Amanda requires its own group and user on both the server and the clients. These are required before installation since the install script will attempt tochown various programs. I tend to pick a matching UID and GID to make
things easy to remember. The actual number isn't meaningful, so I'll randomly
pick 5555 for my example here. The sample configuration called
Full, which I'll provide later, also requires that I create several
files and directories and set their permissions correctly.
groupadd -g 5555 amanda useradd -c amanda -d /usr/local/etc/amanda -m -g 5555 -u 5555 -G sys amanda mkdir -p /usr/local/etc/amanda mkdir -p /var/log/amanda/Full chown -R amanda:amanda /var/log/amanda /usr/local/etc/amanda touch /etc/dumpdates chmod 664 /etc/dumpdates chown root:sys /etc/dumpdates In addition, I must modify echo "amanda 10080/udp # amanda" >> /etc/services echo "amandaidx 10082/tcp # amanda" >> /etc/services echo "amidxtape 10083/tcp # amanda" >> /etc/services echo "amanda dgram udp wait amanda /usr/local/libexec/amandad \ amandad # amanda" >> /etc/inetd.conf echo "amandaidx stream tcp nowait amanda /usr/local/libexec/amindexd \ amindexd # amanda" >> /etc/inetd.conf echo "amidxtape stream tcp nowait amanda /usr/local/libexec/amidxtaped \ amidxtaped # amanda" >> /etc/inetd.conf Lastly, Amanda security requires that I have a file called echo "tapehost.my.domain" > /usr/local/etc/amanda/.amandahosts
Retrieving and Compiling the Source Amanda may require some software, such as readline, that you may or may not already have on your system. Be sure to resolve any software dependencies if you receive errors. Obtain the source code for amanda-2.4.4p1.tar.gz, unpack it in a build directory on your tape host, configure it, and build it:
tar zxf amanda-2.4.4p1.tar.gz
./configure --with-user=amanda --with-group=amanda \
--with-libraries=/usr/local/lib
make
make check
make install
This installs the following programs as well as supporting libraries, man
pages, and information files. These commands have their own man page or are
covered in the main /usr/local/libexec/amandad /usr/local/libexec/driver /usr/local/libexec/calcsize /usr/local/libexec/dumper /usr/local/libexec/killpgrp /usr/local/libexec/planner /usr/local/libexec/rundump /usr/local/libexec/taper /usr/local/libexec/runtar /usr/local/libexec/amcleanupdisk /usr/local/libexec/selfcheck /usr/local/libexec/chg-spectra /usr/local/libexec/sendbackup /usr/local/libexec/chg-scsi /usr/local/libexec/sendsize /usr/local/libexec/chg-manual /usr/local/libexec/versionsuffix /usr/local/libexec/chg-multi /usr/local/libexec/patch-system /usr/local/libexec/chg-mtx /usr/local/libexec/amindexd /usr/local/libexec/chg-rth /usr/local/libexec/amlogroll /usr/local/libexec/chg-chs /usr/local/libexec/amtrmidx /usr/local/libexec/chg-chio /usr/local/libexec/amtrmlog /usr/local/libexec/chg-zd-mtx /usr/local/libexec/amidxtaped /usr/local/sbin/ammt /usr/local/sbin/amcleanup /usr/local/sbin/amdd /usr/local/sbin/amdump /usr/local/sbin/amadmin /usr/local/sbin/amoverview /usr/local/sbin/amcheck /usr/local/sbin/amrmtape /usr/local/sbin/amflush /usr/local/sbin/amtoc /usr/local/sbin/amgetconf /usr/local/sbin/amverify /usr/local/sbin/amlabel /usr/local/sbin/amstatus /usr/local/sbin/amtape /usr/local/sbin/amrestore /usr/local/sbin/amreport /usr/local/sbin/amrecover /usr/local/sbin/amcheckdb Create a Client Package for the Solaris OS Since the client software will reside on a number of machines, it's easiest to create a portable package. On a client machine, configure amanda to build without the server component.
tar zxf amanda-2.4.4p1.tar.gz
./configure --with-user=amanda --with-group=amanda \
--with-libraries=/usr/local/lib --without-server
make
make check
make install
I use the following pkginfo, prototype, preinstall, and postremove files to create the amanda package: amanda-client-2.4.4p1-pkginfo BASEDIR=/usr/local PATH=/sbin:/usr/sbin:/usr/bin:/usr/sadm/install/bin PKG=amandaclient NAME=amanda-client VERSION=2.4.4p1 ARCH=sun4u CATEGORY=application VENDOR=MyDomain EMAIL=support@my.domain CLASSES=none amanda-client-2.4.4p1-prototype i pkginfo=amanda-client-2.4.4p1-pkginfo i preinstall=amanda-client-2.4.4p1-preinstall i postremove=amanda-client-2.4.4p1-postremove !search /usr/local d none lib 0755 root other f none lib/libamanda.la 0755 root other f none lib/libamanda.a 0644 root other f none lib/libamclient.la 0755 root other f none lib/libamclient.a 0644 root other f none lib/libamtape.la 0755 root other f none lib/libamtape.a 0644 root other d none libexec 0755 root other f none libexec/amandad 0755 amanda amanda f none libexec/calcsize 4750 root amanda f none libexec/killpgrp 4750 root amanda f none libexec/rundump 4750 root amanda f none libexec/runtar 4750 root amanda f none libexec/selfcheck 0755 amanda amanda f none libexec/sendbackup 0755 amanda amanda f none libexec/sendsize 0755 amanda amanda f none libexec/versionsuffix 0755 amanda amanda f none libexec/patch-system 0755 amanda amanda f none libexec/amidxtaped 0755 amanda amanda d none sbin 0755 root other f none sbin/ammt 0755 amanda amanda f none sbin/amdd 0755 amanda amanda f none sbin/amrestore 0755 amanda amanda f none sbin/amrecover 0750 amanda amanda d none man 0755 root other d none man/man8 0755 root other f none man/man8/amanda.8 0644 amanda amanda f none man/man8/amrecover.8 0644 amanda amanda f none man/man8/amrestore.8 0644 amanda amanda amanda-client-2.4.4p1-preinstall
#!/bin/sh
# make sure that TMPDIR is on the same filesystem as the password file to
# avoid fatal errors with useradd
TMPDIR=/etc
export TMPDIR
mkdir -p /usr/local/etc/amanda
echo "tapehost.my.domain" > /usr/local/etc/amanda/.amandahosts
touch /etc/dumpdates
chmod 664 /etc/dumpdates
chown root:sys /etc/dumpdates
if [ -f /etc/group ] && [ -f /etc/shadow ]; then
/usr/sbin/groupadd -g 5555 amanda
/usr/sbin/useradd -c amanda -d /usr/local/etc/amanda -g 5555 -u 5555 -G sys amanda
fi
if [ -f /etc/services ]; then
cp /etc/services /etc/services.orig
fi
echo "amanda 10080/udp # amanda" >> /etc/services
echo "amandaidx 10082/tcp # amanda" >> /etc/services
echo "amidxtape 10083/tcp # amanda" >> /etc/services
if [ -f /etc/inetd.conf ]; then
cp /etc/inetd.conf /etc/inetd.conf.orig
fi
echo "amanda dgram udp wait amanda /usr/local/libexec/amandad amandad" \
>> /etc/inetd.conf
amanda-client-2.4.4p1-postremove #!/bin/sh TMPDIR=/etc export TMPDIR if [ -f /etc/group ] && [ -f /etc/shadow ]; then /usr/sbin/usermod -G , amanda /usr/sbin/groupdel amanda /usr/sbin/userdel amanda fi if [ -f /etc/inetd.conf ]; then grep -vi amanda /etc/inetd.conf > /etc/inetd.$$ mv /etc/inetd.$$ /etc/inetd.conf fi if [ -f /etc/services ]; then grep -vi amanda /etc/services > /etc/services.$$ mv /etc/services.$$ /etc/services fi Now create the package and install it on each client: pkgmk -b /usr/local -o -f amanda-client-2.4.4p1-prototype cd /var/spool/pkg; pkgtrans -s `pwd` /tmp/amandaclient-2.4.4p1 pkgadd -d /tmp/amandaclient-2.4.4p1 Also make sure that the user amanda has read access on all raw disk devices that it needs to back up. Group sys read is sufficient since amanda is in group sys:
crw-r----- 1 root sys 136, 0 May 27 18:03 \
/devices/pci@1f,0/pci@1,1/ide@3/dad@0,0:a,raw
crw-r----- 1 root sys 136, 3 Aug 19 13:54 \
/devices/pci@1f,0/pci@1,1/ide@3/dad@0,0:d,raw
crw-r----- 1 root sys 136, 4 Aug 19 13:54 \
/devices/pci@1f,0/pci@1,1/ide@3/dad@0,0:e,raw
crw-r----- 1 root sys 136, 5 Aug 19 13:54 \
/devices/pci@1f,0/pci@1,1/ide@3/dad@0,0:f,raw
Sample ConfigurationThis sample configuration performs full backups of each data set every time
amanda runs. Amanda runs from cron twice a week, and the dumps are expired
after three weeks. There are 12 AIT2 tapes in the cycle, and each run can
take up to two tapes. The tapes are housed in a Spectra Logic tape changer,
controlled by the program First, I make the directory for the mkdir -p /usr/local/etc/amanda/Full In that directory, I put the file
# amanda.conf - Amanda configuration file for full dumps
org "MyDomain" # your organization name for reports
mailto "operator@my.domain" # space separated list of operators at your site
dumpuser "amanda" # the user to run dumps under
inparallel 15 # maximum dumpers that will run in parallel (max 63)
# this maximum can be increased at compile-time,
# modifying MAX_DUMPERS in server-src/driverio.h
dumporder "ttBTBTBTBTBTBTB" # specify the priority order of each dumper
# s -> smallest size
# S -> biggest size
# t -> smallest time
# T -> biggest time
# b -> smallest bandwidth
# B -> biggest bandwidth
# try "BTBTBTBTBTBT" if you are not holding
# disk constrained
netusage 10000000 Kbps # maximum net bandwidth for Amanda, in KB per sec
dumpcycle 3 weeks # the number of days in the normal dump cycle
runspercycle 8 # the number of amdump runs in dumpcycle days
# (4 weeks * 5 amdump runs per week -- just weekdays)
tapecycle 12 tapes # the number of tapes in rotation
# 4 weeks (dumpcycle) times 5 tapes per week (just
# the weekdays) plus a few to handle errors that
# need amflush and so we do not overwrite the full
# backups performed at the beginning of the previous
# cycle
bumpsize 20 Mb # minimum savings (threshold) to bump level 1 -> 2
bumpdays 1 # minimum days at each level
bumpmult 4 # threshold = bumpsize * bumpmult^(level-1)
etimeout 600 # number of seconds per filesystem for estimates.
#etimeout -600 # total number of seconds for estimates.
# a positive number will be multiplied by the number of filesystems on
# each host; a negative number will be taken as an absolute total time-out.
# The default is 5 minutes per filesystem.
dtimeout 1800 # number of idle seconds before a dump is aborted.
ctimeout 30 # maximum number of seconds that amcheck waits
# for each client host
tapebufs 20
# A positive integer telling taper how many 32k buffers to allocate.
# WARNING! If this is set too high, taper will not be able to allocate
# the memory and will die. The default is 20 (640k).
# Specify tape device and/or tape changer. If you don't have a tape
# changer, and you don't want to use more than one tape per run of
# amdump, just comment out the definition of tpchanger.
tpchanger "chg-spectra" # the tape-changer glue script
tapedev "/dev/rmt/0cn" # the no-rewind tape device to be used
changerfile "/usr/local/etc/amanda/changer"
changerdev "/dev/scsi/changer/c1t4d0"
runtapes 1 # number of tapes to be used in a single run of amdump
tapetype SDX2-50C # what kind of tape it is (see tapetypes below)
labelstr "^Full-[0-9][0-9]*$" # label constraint regex: all tapes must match
# Specify holding disks. These are used as a temporary staging area for
# dumps before they are written to tape and are recommended for most sites.
# If no holding disks are specified then all dumps will be written directly
# to tape. If a dump is too big to fit on the holding disk then it will be
# written directly to tape. If more than one holding disk is specified then
# they will all be used based on activity and available space.
holdingdisk hd1 {
comment "main holding disk"
directory "/data/amanda/Full" # where the holding disk is
use 65 Gb # how much space can we use on it
}
# If amanda cannot find a tape on which to store backups, it will run
# as many backups as it can to the holding disks. In order to save
# space for unattended backups, by default, amanda will only perform
# incremental backups in this case, i.e., it will reserve 100% of the
# holding disk space for the so-called degraded mode backups.
# However, if you specify a different value for the `reserve'
# parameter, amanda will not degrade backups if they will fit in the
# non-reserved portion of the holding disk.
# reserve 30 # percent
# This means save at least 30% of the holding disk space for degraded
# mode backups.
reserve 0
autoflush no
# if autoflush is set to yes, then amdump will schedule all dump on
# holding disks to be flush to tape during the run.
# The format for a ColumnSpec is a ',' seperated list of triples.
# Each triple consists of
# + the name of the column (as in ColumnNameStrings)
# + prefix before the column
# + the width of the column, if set to -1 it will be recalculated
# to the maximum length of a line to print.
#
# You need only specify those columns that should be changed from
# the default. If nothing is specified in the configfile, the
# above compiled in values will be in effect, resulting in an
# output as it was all the time.
# The names of the columns are:
# HostName, Disk, Level, OrigKB, OutKB, Compress, DumpTime, DumpRate,
# TapeTime and TapeRate.
# ElB, 1999-02-24.
# columnspec "Disk=1:18,HostName=0:10,OutKB=1:7"
# (Note - the line below should be on one line)
columnspec "HostName=0:12,Disk=1:12,Level=1:1,OrigKB=1:7,OutKB=1:7,
DumpTime=1:6,DumpRate=1:7,TapeTime=1:6,TapeRate=1:7,Compress=1:0"
# Amanda needs a few Mb of diskspace for the log and debug files,
# as well as a database. This stuff can grow large, so the conf directory
# isn't usually appropriate. Some sites use /usr/local/var and some /usr/adm.
# Create an amanda directory under there. You need a separate infofile and
# logdir for each configuration, so create subdirectories for each conf and
# put the files there. Specify the locations below.
# Note that, although the keyword below is infofile, it is only so for
# historic reasons, since now it is supposed to be a directory (unless
# you have selected some database format other than the `text' default)
infofile "/var/log/amanda/Full/curinfo" # database DIRECTORY
logdir "/var/log/amanda/Full" # log directory
indexdir "/var/log/amanda/Full/index" # index directory
# tapetypes
# Define the type of tape you use here, and use it in "tapetype"
# above. The tapetype tells amanda how many MB will fit on the tape,
# how big the filemarks are, and how fast the tape device is.
# A filemark is the amount of wasted space every time a tape section
# ends. If you run `make tapetype' in tape-src, you'll get a program
# that generates tapetype entries, but it is slow as hell, use it only
# if you really must and, if you do, make sure you post the data to
# the amanda mailing list, so that others can use what you found out
# by searching the archives.
define tapetype SDX2-50C {
comment "AIT2"
length 50000 mbytes
filemark 3120 kbytes
speed 5371 kps
}
# dumptypes - define how a filesystem is backed up
define dumptype global {
# This is quite useful for setting global parameters, so you don't have
# to type them everywhere. All dumptype definitions in this sample file
# do include these definitions, either directly or indirectly.
comment "Global definitions"
compress NONE
priority high
dumpcycle 0
holdingdisk yes
index yes
record yes
}
define dumptype always-full {
global
comment "Full dump of this filesystem always"
index yes
program "DUMP"
maxdumps 2
}
define dumptype always-full-tar {
global
comment "Full backups using tar"
index yes
program "GNUTAR"
maxdumps 4
}
# define usable network interfaces
define interface local {
comment "a local disk"
use 10000 kbps
}
define interface le0 {
comment "10 Mbps ethernet"
use 9000 kbps
}
define interface hme0 {
comment "100 Mbps ethernet"
use 900000 kbps
}
More information on the The following goes in solaris8.my.domain / always-full solaris8.my.domain /usr always-full solaris8.my.domain /var always-full solaris8.my.domain /local always-full solaris9.my.domain c0t0d0s0 always-full solaris9.my.domain c0t0d0s3 always-full solaris9.my.domain c0t0d0s4 always-full solaris9.my.domain c0t0d0s5 always-full freebsd.my.domain ad0s1a always-full freebsd.my.domain ad0s1e always-full freebsd.my.domain ad0s1f always-full freebsd.my.domain /raid always-full-tar freebsd.my.domain //windows2000/work always-full-tar freebsd.my.domain //windows2000/doc_share always-full-tar osx.my.domain /Applications always-full-tar osx.my.domain /Developer always-full-tar osx.my.domain /Library always-full-tar osx.my.domain /Network always-full-tar osx.my.domain /System always-full-tar osx.my.domain /Users always-full-tar osx.my.domain /bin always-full-tar osx.my.domain /dev always-full-tar osx.my.domain /private always-full-tar osx.my.domain /root always-full-tar osx.my.domain /sbin always-full-tar osx.my.domain /sw always-full-tar osx.my.domain /usr always-full-tar Labeling TapesNow that the labelstr "^Full-[0-9][0-9]*$" # label constraint regex: all tapes must match
I must now match that regular expression when labeling my tapes, or amanda
will not know that the tapes belong with the amlabel Full Full-01 slot 1 amlabel Full Full-02 slot 2 amlabel Full Full-03 slot 3 amlabel Full Full-04 slot 4 Running 0 Full-01 reuse 0 Full-02 reuse 0 Full-03 reuse 0 Full-04 reuse Performing BackupsNow that my configuration is complete and I've labeled some tapes, I can
actually run a backup. First I run the amcheck Full Amanda Tape Server Host Check ----------------------------- Holding disk /data/amanda/Full: 69770037 KB disk space available, that's plenty amcheck-server: slot 1: date 20030813 label Full-01 (exact label match) NOTE: skipping tape-writable test Tape Full-01 label ok Server check took 46.133 seconds Amanda Backup Client Hosts Check -------------------------------- Client check: 4 hosts checked in 1.108 seconds, 0 problems found The check shows that everything is ok and ready for a dump, so I run the dump as the user amanda: amdump Full Once the dump finishes, a report will be emailed to operator@my.domain, as
specified in the From: amanda To automate backups, amanda has the following crontab. Running
0 20 * * 2,5 /usr/local/sbin/amcheck Full
Restoring Files and Data SetsAmanda can restore selected files or entire data sets. The program
Here's an example from the
EXAMPLES
The following does an interactive restore of disk rz3g from
host seine, to restore particular files. Note the use of
the b option to restore, which causes it to read in units of
two 512-byte blocks (1 Kbyte) at a time. This helps keep it
from complaining about short reads.
% amrestore -p /dev/nrmt9 seine rz3g | restore -ivbf 2 -
The next example extracts all backup images for host seine.
This is the usual way to extract all data for a host after a
disk crash.
% amrestore /dev/nrmt9 seine
Here's an example from the
EXAMPLES
The following shows the recovery of an old syslog file.
# cd /var/log
# ls -l syslog.7
syslog.7: No such file or directory
# amrecover
AMRECOVER Version 2.4.4p1. Contacting server on
# tapehost.my.domain
220 tapehost.my.domain AMANDA index server (2.4.4p1) ready.
Setting restore date to today (1997-12-09)
200 Working date set to 1997-12-09.
200 Config set to DailySet1.
200 Dump host set to this-host.some.org.
$CWD '/var/log' is on disk '/var' mounted at '/var'.
200 Disk set to /var.
/var/log
WARNING: not on root of selected filesystem, check man-page!
amrecover> ls
1997-12-09 daemon.log
1997-12-09 syslog
1997-12-08 authlog
1997-12-08 sysidconfig.log
1997-12-08 syslog.0
1997-12-08 syslog.1
1997-12-08 syslog.2
1997-12-08 syslog.3
1997-12-08 syslog.4
1997-12-08 syslog.5
1997-12-08 syslog.6
1997-12-08 syslog.7
amrecover> add syslog.7
Added /log/syslog.7
amrecover> lpwd
/var/log
amrecover> lcd ..
/var
amrecover> extract
Extracting files using tape drive /dev/null on host
tapehost.my.domain
The following tapes are needed: DMP014
Restoring files into directory /var
Continue? [Y/n]: y
Load tape DMP014 now
Continue? [Y/n]: y
set owner/mode for '.'? [yn] n
amrecover> quit
200 Good bye.
# ls -l syslog.7
total 26
-rw-r--r-- 1 root other 12678 Oct 14 16:36 syslog.7
ResourcesFor further information, see the following resources:
Unless otherwise licensed, code in all technical manuals herein (including articles, FAQs, samples) is provided under this License. |
|