The 15-Minute Backup Solution
Secure Network Backups in a Heterogeneous Environment in the Time it Takes to Have Pizza Delivered (All Using Open Source Software!)
This setup below was performed using Amanda 2.5.1p2. To learn how to set up:
-
- the latest version of Amanda 3.x with new configuration tools
-
- the new Volume Shadow Copy Service (VSS) based Zmanda Windows Client Community Edition
Please register on Zmanda Network and read
Setting-Up an Open Source Backup Software Amanda Community in About 15 Minutes white paper available in the Resources section.
Today’s businesses rarely
run on just one operating system. Linux users and administrators often
have strong preferences for one distribution over another; web
designers might lean towards the Mac; legacy software and hardware can
include various UNIX operating systems. Despite the complexity of
modern business computing environments, a system administrator is
expected to find a reliable backup solution.
Even in the case where
users are expected to keep important files on networked resources, for
true intellectual data security, desktop machines and laptops will also
be backed up. The price of hard disk storage is continuously falling,
bringing terabytes of storage within reach, and increasing the amount
of data that can potentially be lost. (The amount of data that you have
will always expand to fit the storage available; as the golden rule
states.) We live in a global and e-commerce economy, where businesses
run around the clock and crucial business data changes commensurately.
The Challenge
For our 15-minute challenge, you will backup two Linux systems (each running a different Linux distribution) and one Windows system, using freely downloadable open source software.Our scenario is as follows:
The user "pavel" works with sensitive information. We need to make an encrypted backup of his home directory, /home/pavel, which resides on a Fedora Core Linux system called Iron. Our webmaster needs the webserver's document home backed up, the /var/www/html directory on a SUSE Enterprise Linux system called Copper. Our manager works solely on a Windows XP system called Uranium, and keeps all of his work in the MyDocuments folder, so we will need to add //Uranium/MyDocuments to our backup configuration.
The Solution: Amanda
Amanda is open source
backup software that is flexible, secure and scalable to dynamic
computing environments. Amanda can save you from expensive proprietary
backup software and those custom backup scripts that have a propensity
to break at the worst times. Dating back to 1991, Amanda has been used
successfuly in environments from one standalone machine to hundreds of
clients. Amanda is so thoroughly documented, from community wikis to
published system administration texts, that it might be hard to discern
just how easy an Amanda backup can be.
This article will show
you how, in about 15 minutes, you can:
1. Install and configure
the Amanda backup server.
2. Prepare three
different clients for backup.
3. Set backup parameters.
4. Verify the
configuration.
5. Verify the backup.
We will install and
configure Amanda backup server software on Quartz, which is
running Red Hat Enterprise Linux. We will install and configure Amanda
backup client software on Copper and on Iron. The Windows XP client,
Uranium, will be backed up with Amanda server software running in
conjunction with Samba on the backup server, Quartz.
-
ClientFilesystemOSCompressionEncryptionCopper/var/www/htmlSLES9YesNoIron/home/pavelFC4YesYesUranium//uranium/MyDocuments*WINXPYesNo* using Samba (i.e. without installing any software on the Windows system)
Prerequisites
The basic Amanda
setup consists of an Amanda server, the Amanda client or clients that
are to be backed up, and the backup storage media such as a tape or
hard disk device. An intermediate holding area for caching data is not
absolutely necessary, but will improve performance significantly and is
considered part of a basic setup.
Before we begin,
please review the introduction to
Amanda. Then, note the following prerequisites:
-
tar 1.15 or later and xinetd are installed on Quartz, Iron and Copper.
-
Quartz is able to send mail to the root user.
-
The systems are all on the same network and available.
-
You have root access, and root access through SSH is enabled and working.
-
The directories to be backed up exist.
-
The Amanda 2.5.1p2 backup_server RPM should be available on Quartz, and the backup_client RPM should be available on Iron and Copper. Amanda binary and source RPM packages and source tarballs are freely available from Zmanda.
-
Quartz, the backup server, is running Samba client software. Samba is also freely available open source software.
To support the
encrypted backup of /home/pavel on Iron, the following packages
should be installed and available on Iron:
-
sharutils (for the tool uuencode)
Also note that
this article assumes a fresh install of Amanda. If you have an existing
Amanda installation, additional steps are needed to ensure the proper
upgrade to the latest Amanda release, (2.5.1p2 and
later).
TIP: You can
copy and paste all of the examples here, making appropriate
modifications for your environment.
Order Pizza
Call your
favorite pizza delivery place, set your stopwatch and...
Install and Configure the Amanda Backup Server
1. Log
in as root on Quartz, the Red Hat Enterprise Linux 4 server.
2. Install
the Amanda 2.5.1p2 amanda-backup_server RPM. Installing the package
also
creates a user named amandabackup who
belongs to the group disk.
[root@quartz
server]# rpm -ivh amanda-backup_server-2.5.1p2-1.rhel4.i386.rpm
warning:
amanda-backup_server-2.5.1p2-1.rhel4.i386.rpm: V3 DSA signature: NOKEY,
key ID
3c5d1c92
Preparing... ########################################### [100%]
Jan 5 2007 12:12:55: Preparing to install: Amanda Community Edition - version 2.5.1p2
Jan 5 2007 12:12:55: Checking for 'amandabackup' user...
Jan 5 2007 12:12:55:
Jan 5 2007 12:12:55: The Amanda backup software is configured to operate as the
Jan 5 2007 12:12:55: user 'amandabackup'. This user exists on your system and has not
Jan 5 2007 12:12:55: been modified. To ensure that Amanda functions properly,
Jan 5 2007 12:12:56: please see that the following parameters are set for that
Jan 5 2007 12:12:56: user.:
Jan 5 2007 12:12:56:
Jan 5 2007 12:12:56: SHELL: /bin/sh
Jan 5 2007 12:12:56: HOME: /var/lib/amanda
Jan 5 2007 12:12:56: Default group: disk
Jan 5 2007 12:12:56:
Jan 5 2007 12:12:56: Checking ownership of '/var/lib/amanda'... correct.
Jan 5 2007 12:12:57:
Jan 5 2007 12:12:57: === Amanda backup server installation started. ===
Preparing... ########################################### [100%]
Jan 5 2007 12:12:55: Preparing to install: Amanda Community Edition - version 2.5.1p2
Jan 5 2007 12:12:55: Checking for 'amandabackup' user...
Jan 5 2007 12:12:55:
Jan 5 2007 12:12:55: The Amanda backup software is configured to operate as the
Jan 5 2007 12:12:55: user 'amandabackup'. This user exists on your system and has not
Jan 5 2007 12:12:55: been modified. To ensure that Amanda functions properly,
Jan 5 2007 12:12:56: please see that the following parameters are set for that
Jan 5 2007 12:12:56: user.:
Jan 5 2007 12:12:56:
Jan 5 2007 12:12:56: SHELL: /bin/sh
Jan 5 2007 12:12:56: HOME: /var/lib/amanda
Jan 5 2007 12:12:56: Default group: disk
Jan 5 2007 12:12:56:
Jan 5 2007 12:12:56: Checking ownership of '/var/lib/amanda'... correct.
Jan 5 2007 12:12:57:
Jan 5 2007 12:12:57: === Amanda backup server installation started. ===
1:amanda-backup_server ###########################################
[100%]
Jan 5 2007
12:13:05: Updating system library cache...done.
Jan 5 2007 12:13:21: Installing '/etc/amandates'.
Jan 5 2007 12:13:21: The file '/etc/amandates' has been created.
Jan 5 2007 12:13:21: Ensuring correct permissions for '/etc/amandates'.
Jan 5 2007 12:13:21: '/etc/amandates' Installation successful.
Jan 5 2007 12:13:22: Checking '/var/lib/amanda/.amandahosts' file.
Jan 5 2007 12:13:22: Checking for '/var/lib/amanda/.profile' and ensuring correct environment.
Jan 5 2007 12:13:23: Setting ownership and permissions for '/var/lib/amanda/.profile'
Jan 5 2007 12:13:23: === Amanda backup server installation complete. ===
Amanda installation log can be found in '/var/log/amanda/install.log' and errors (if any) in '/var/log/amanda/install.err'.
Jan 5 2007 12:13:21: Installing '/etc/amandates'.
Jan 5 2007 12:13:21: The file '/etc/amandates' has been created.
Jan 5 2007 12:13:21: Ensuring correct permissions for '/etc/amandates'.
Jan 5 2007 12:13:21: '/etc/amandates' Installation successful.
Jan 5 2007 12:13:22: Checking '/var/lib/amanda/.amandahosts' file.
Jan 5 2007 12:13:22: Checking for '/var/lib/amanda/.profile' and ensuring correct environment.
Jan 5 2007 12:13:23: Setting ownership and permissions for '/var/lib/amanda/.profile'
Jan 5 2007 12:13:23: === Amanda backup server installation complete. ===
Amanda installation log can be found in '/var/log/amanda/install.log' and errors (if any) in '/var/log/amanda/install.err'.
3. The
Amanda services are started by the extended internet daemon, xinetd, which is why you must have xinetd installed on every Amanda server
and client. In any text editor, create one xinetd startup file, /etc/xinetd.d/amandaserver , with content as follows.
For the /etc/xinetd.d/amandaserver file, on Quartz:
#
default: on
#
# description: Amanda services for Amanda server and client.
#
service amanda
{
disable = no
socket_type = stream
protocol = tcp
wait = no
user = amandabackup
group = disk
groups = yes
server = /usr/lib/amanda/amandad
server_args = -auth=bsdtcp amdump amindexd amidxtaped
}
#
# description: Amanda services for Amanda server and client.
#
service amanda
{
disable = no
socket_type = stream
protocol = tcp
wait = no
user = amandabackup
group = disk
groups = yes
server = /usr/lib/amanda/amandad
server_args = -auth=bsdtcp amdump amindexd amidxtaped
}
4. Restart xinetd on Quartz.
[root@quartz
xinetd.d]# service xinetd reload
Reloading configuration: [ OK ]
Reloading configuration: [ OK ]
5. Note
the time. Only about five minutes should have passed!
Install and Configure Three Different Amanda Clients
Installation
of Amanda Client RPM on Iron (FC4)
1. Log
in as root on Iron, your Fedora Core 4 client.
2. Install
the Amanda 2.5.1p2 backup_client RPM. Installing the package also
creates a
user named amandabackup who belongs
to the group disk.
[root@iron
client]# rpm -ivh
amanda-backup_client-2.5.1p2-1.fc4.i386.rpm
warning: amanda-backup_client-2.5.1p2-1.fc4.i386.rpm: Header V3 DSA signature: NOKEY, key ID 3c5d1c92
Preparing... ########################################### [100%]
Jan 5 2007 10:17:16: Preparing to install: Amanda Community Edition - version 2.5.1p2
Jan 5 2007 10:17:16: Checking for 'amandabackup' user...
Jan 5 2007 10:17:16:
Jan 5 2007 10:17:16: The Amanda backup software is configured to operate as the
Jan 5 2007 10:17:17: user 'amandabackup'. This user exists on your system and has not
Jan 5 2007 10:17:17: been modified. To ensure that Amanda functions properly,
Jan 5 2007 10:17:17: please see that the following parameters are set for that
Jan 5 2007 10:17:17: user.:
Jan 5 2007 10:17:17:
Jan 5 2007 10:17:17: SHELL: /bin/sh
Jan 5 2007 10:17:17: HOME: /var/lib/amanda
Jan 5 2007 10:17:17: Default group: disk
Jan 5 2007 10:17:17:
Jan 5 2007 10:17:17: Checking ownership of '/var/lib/amanda'... correct.
Jan 5 2007 10:17:17:
Jan 5 2007 10:17:17: === Amanda backup client installation started. ===
1:amanda-backup_client ########################################### [100%]
warning: amanda-backup_client-2.5.1p2-1.fc4.i386.rpm: Header V3 DSA signature: NOKEY, key ID 3c5d1c92
Preparing... ########################################### [100%]
Jan 5 2007 10:17:16: Preparing to install: Amanda Community Edition - version 2.5.1p2
Jan 5 2007 10:17:16: Checking for 'amandabackup' user...
Jan 5 2007 10:17:16:
Jan 5 2007 10:17:16: The Amanda backup software is configured to operate as the
Jan 5 2007 10:17:17: user 'amandabackup'. This user exists on your system and has not
Jan 5 2007 10:17:17: been modified. To ensure that Amanda functions properly,
Jan 5 2007 10:17:17: please see that the following parameters are set for that
Jan 5 2007 10:17:17: user.:
Jan 5 2007 10:17:17:
Jan 5 2007 10:17:17: SHELL: /bin/sh
Jan 5 2007 10:17:17: HOME: /var/lib/amanda
Jan 5 2007 10:17:17: Default group: disk
Jan 5 2007 10:17:17:
Jan 5 2007 10:17:17: Checking ownership of '/var/lib/amanda'... correct.
Jan 5 2007 10:17:17:
Jan 5 2007 10:17:17: === Amanda backup client installation started. ===
1:amanda-backup_client ########################################### [100%]
Jan 5 2007
10:17:21: Updating system library cache...done.
Jan 5 2007 10:17:30: Checking '/var/lib/amanda/.amandahosts' file.
Jan 5 2007 10:17:31: Checking for '/var/lib/amanda/.profile' and ensuring correct environment.
Jan 5 2007 10:17:31: Setting ownership and permissions for '/var/lib/amanda/.profile'
Jan 5 2007 10:17:31: Checking for '/var/lib/amanda/.profile' and ensuring correct environment.
Jan 5 2007 10:17:31: Setting ownership and permissions for '/var/lib/amanda/.profile'
Jan 5 2007 10:17:31: === Amanda backup client installation complete. ===
Amanda installation log can be found in '/var/log/amanda/install.log' and errors (if any) in '/var/log/amanda/install.err'.
Jan 5 2007 10:17:30: Checking '/var/lib/amanda/.amandahosts' file.
Jan 5 2007 10:17:31: Checking for '/var/lib/amanda/.profile' and ensuring correct environment.
Jan 5 2007 10:17:31: Setting ownership and permissions for '/var/lib/amanda/.profile'
Jan 5 2007 10:17:31: Checking for '/var/lib/amanda/.profile' and ensuring correct environment.
Jan 5 2007 10:17:31: Setting ownership and permissions for '/var/lib/amanda/.profile'
Jan 5 2007 10:17:31: === Amanda backup client installation complete. ===
Amanda installation log can be found in '/var/log/amanda/install.log' and errors (if any) in '/var/log/amanda/install.err'.
3. In
any text editor, create an xinetd startup file, /etc/xinetd.d/amandaclient, with content as follows.
#
default: on
#
# description: Amanda services for Amanda client.
#
service amanda
{
disable = no
socket_type = stream
protocol = tcp
wait = no
user = amandabackup
group = disk
groups = yes
server = /usr/lib/amanda/amandad
server_args = -auth=bsdtcp amdump
}
#
# description: Amanda services for Amanda client.
#
service amanda
{
disable = no
socket_type = stream
protocol = tcp
wait = no
user = amandabackup
group = disk
groups = yes
server = /usr/lib/amanda/amandad
server_args = -auth=bsdtcp amdump
}
4. Restart xinetd on Iron.
[root@ironxinetd.d]#
service xinetd reload
Reloading configuration: [ OK ]
Reloading configuration: [ OK ]
5. Become
the amandabackup user and append the
line "quartz.zmanda.com amandabackup amdump" to the /var/lib/amanda/.amandahosts file on
Iron. This allows Quartz, the Amanda backup server, to connect to Iron,
the
Amanda client.
Note that
you
should use fully qualified domain names when configuring Amanda.
-bash-3.00$
echo quartz.zmanda.com amandabackup
amdump >> /var/lib/amanda/.amandahosts
-bash-3.00$ chmod 700 /var/lib/amanda/.amandahosts
-bash-3.00$ chmod 700 /var/lib/amanda/.amandahosts
6. Save
the passphrase as a hidden file in the home directory of the amandabackup user. Protect the file with
the proper permissions.
As the
user amandabackup:
-sh-3.00$
chown amandabackup:disk ~amandabackup/.am_passphrase
-sh-3.00$ chmod 700 ~amandabackup/.am_passphrase
-sh-3.00$ chmod 700 ~amandabackup/.am_passphrase
7. Create
a script that enables encryption on the client Iron.
As root create a file /usr/sbin/amcryptsimple:
#!/usr/bin/perl
-w
use Time::Local;
my $AMANDA='amandabackup';
$AMANDA_HOME = (getpwnam($AMANDA) )[7] || die "Cannot find $AMANDA home directory\n" ;
$AM_PASS = "$AMANDA_HOME/.am_passphrase";
$ENV{'PATH'} = '/usr/local/bin:/usr/bin:/usr/sbin:/bin:/sbin';
$ENV{'GNUPGHOME'} = "$AMANDA_HOME/.gnupg";
sub encrypt() {
system "gpg --batch --disable-mdc --symmetric --cipher-algo AES256 --passphrase-fd 3 3<$AM_PASS";}
sub decrypt() {
system "gpg --batch --quiet --no-mdc-warning --decrypt --passphrase-fd 3 3<$AM_PASS";
}
if ( $#ARGV > 0 ) {
die "Usage: $0 [-d]\n";
}
if ( $#ARGV==0 && $ARGV[0] eq "-d" ) {
decrypt();
}
else {
encrypt();
}
use Time::Local;
my $AMANDA='amandabackup';
$AMANDA_HOME = (getpwnam($AMANDA) )[7] || die "Cannot find $AMANDA home directory\n" ;
$AM_PASS = "$AMANDA_HOME/.am_passphrase";
$ENV{'PATH'} = '/usr/local/bin:/usr/bin:/usr/sbin:/bin:/sbin';
$ENV{'GNUPGHOME'} = "$AMANDA_HOME/.gnupg";
sub encrypt() {
system "gpg --batch --disable-mdc --symmetric --cipher-algo AES256 --passphrase-fd 3 3<$AM_PASS";}
sub decrypt() {
system "gpg --batch --quiet --no-mdc-warning --decrypt --passphrase-fd 3 3<$AM_PASS";
}
if ( $#ARGV > 0 ) {
die "Usage: $0 [-d]\n";
}
if ( $#ARGV==0 && $ARGV[0] eq "-d" ) {
decrypt();
}
else {
encrypt();
}
7. Change
the owership and the permissions on the file /usr/sbin/amcryptsimple you just created:
[root@iron
sbin]# chown amandabackup:disk
/usr/sbin/amcryptsimple
[root@iron sbin]# chmod 750 /usr/sbin/amcryptsimple
[root@iron sbin]# chmod 750 /usr/sbin/amcryptsimple
9. This
completes configuration of the Amanda client on Iron.
Installation
of Amanda Client RPM on Copper (SLES9)
1. Log
in as the root user on Copper, your
SUSE Linux Enterprise Server 9 client.
2. Install
the Amanda 2.5.1p2 backup_client RPM. Installing the package also
creates a
user named amandabackup who belongs
to the group disk.
copper:/
# rpm -ivh
amanda-backup_client-2.5.1p2-1.sles9.i586.rpm
warning: amanda-backup_client-2.5.1p2-1.sles9.i586.rpm: V3 DSA signature: NOKEY, key ID 3c5d1c92
warning: amanda-backup_client-2.5.1p2-1.sles9.i586.rpm: V3 DSA signature: NOKEY, key ID 3c5d1c92
Preparing... ########################################### [100%]
Jan 5 2007
07:20:21: Preparing to install: Amanda Community Edition - version
2.5.1p2
Jan 5 2007 07:20:21: Checking for 'amandabackup' user...
Jan 5 2007 07:20:21:
Jan 5 2007 07:20:21: The Amanda backup software is configured to operate as the
Jan 5 2007 07:20:21: user 'amandabackup'. This user exists on your system and has not
Jan 5 2007 07:20:21: been modified. To ensure that Amanda functions properly,
Jan 5 2007 07:20:21: please see that the following parameters are set for that
Jan 5 2007 07:20:22: user.:
Jan 5 2007 07:20:22:
Jan 5 2007 07:20:22: SHELL: /bin/sh
Jan 5 2007 07:20:22: HOME: /var/lib/amanda
Jan 5 2007 07:20:22: Default group: disk
Jan 5 2007 07:20:22:
Jan 5 2007 07:20:22: Checking ownership of '/var/lib/amanda'... correct.
Jan 5 2007 07:20:22:
Jan 5 2007 07:20:22: === Amanda backup client installation started. ===
1:amanda-backup_client ########################################### [100%]
Jan 5 2007 07:20:21: Checking for 'amandabackup' user...
Jan 5 2007 07:20:21:
Jan 5 2007 07:20:21: The Amanda backup software is configured to operate as the
Jan 5 2007 07:20:21: user 'amandabackup'. This user exists on your system and has not
Jan 5 2007 07:20:21: been modified. To ensure that Amanda functions properly,
Jan 5 2007 07:20:21: please see that the following parameters are set for that
Jan 5 2007 07:20:22: user.:
Jan 5 2007 07:20:22:
Jan 5 2007 07:20:22: SHELL: /bin/sh
Jan 5 2007 07:20:22: HOME: /var/lib/amanda
Jan 5 2007 07:20:22: Default group: disk
Jan 5 2007 07:20:22:
Jan 5 2007 07:20:22: Checking ownership of '/var/lib/amanda'... correct.
Jan 5 2007 07:20:22:
Jan 5 2007 07:20:22: === Amanda backup client installation started. ===
1:amanda-backup_client ########################################### [100%]
Jan 5 2007
07:20:26: Updating system library cache...done.
Jan 5 2007 07:20:26: Checking '/var/lib/amanda/.amandahosts' file.
Jan 5 2007 07:20:27: Checking for '/var/lib/amanda/.profile' and ensuring correct environment.
Jan 5 2007 07:20:27: Setting ownership and permissions for '/var/lib/amanda/.profile'
Jan 5 2007 07:20:27: Checking for '/var/lib/amanda/.profile' and ensuring correct environment.
Jan 5 2007 07:20:27: Setting ownership and permissions for '/var/lib/amanda/.profile'
Jan 5 2007 07:20:27: === Amanda backup client installation complete. ===
Amanda installation log can be found in '/var/log/amanda/install.log' and errors (if any) in '/var/log/amanda/install.err'.
Jan 5 2007 07:20:26: Checking '/var/lib/amanda/.amandahosts' file.
Jan 5 2007 07:20:27: Checking for '/var/lib/amanda/.profile' and ensuring correct environment.
Jan 5 2007 07:20:27: Setting ownership and permissions for '/var/lib/amanda/.profile'
Jan 5 2007 07:20:27: Checking for '/var/lib/amanda/.profile' and ensuring correct environment.
Jan 5 2007 07:20:27: Setting ownership and permissions for '/var/lib/amanda/.profile'
Jan 5 2007 07:20:27: === Amanda backup client installation complete. ===
Amanda installation log can be found in '/var/log/amanda/install.log' and errors (if any) in '/var/log/amanda/install.err'.
3. In
any text editor, create an xinetd startup file, /etc/xinetd.d/amandaclient, with content as follows.
#
default: on
#
# description: Amanda services for Amanda client.
#
service amanda
{
disable = no
socket_type = stream
protocol = tcp
wait = no
user = amandabackup
group = disk
groups = yes
server = /usr/lib/amanda/amandad
server_args = -auth=bsdtcp amdump
}
#
# description: Amanda services for Amanda client.
#
service amanda
{
disable = no
socket_type = stream
protocol = tcp
wait = no
user = amandabackup
group = disk
groups = yes
server = /usr/lib/amanda/amandad
server_args = -auth=bsdtcp amdump
}
5. Restart xinetd on Copper.
copper:/
# /etc/rc.d/xinetd restart
Reload INET services (xinetd). done
Reload INET services (xinetd). done
6. Become
the amandabackup user and append the
line "quartz.zmanda.com amandabackup amdump" to the /var/lib/amanda/.amandahosts file on
Copper. This allows Quartz, the Amanda backup server, to connect to
Copper, the
Amanda client.
Note that you should use fully qualified domain names when configuring Amanda.
Note that you should use fully qualified domain names when configuring Amanda.
-bash-3.00$
echo quartz.zmanda.com amandabackup
amdump >> /var/lib/amanda/.amandahosts
-bash-3.00$ chmod 700 /var/lib/amanda/.amandahosts
-bash-3.00$ chmod 700 /var/lib/amanda/.amandahosts
7. This
completes configuration of the Amanda client on Copper. If you check
your
watch, you should find that only about ten minutes have passed!
Configurations Required to Backup Windows Client Uranium
· Configuration
done on backup server Quartz:
1. The
file /etc/amandapass must be created
manually, owned by the amandabackup user and have permissions of 700. The amandapass file contains share name to user name, password and workgroup mapping.
As the root user:
[root@quartz
/]# echo //uranium/MyDocuments
zmanda%amanda Workgroup >> /etc/amandapass
2. Change
the ownership and permissions on this file:
[root@quartz
etc]# chown amandabackup:disk
/etc/amandapass
[root@quartz etc]# chmod 700 /etc/amandapass
[root@quartz etc]# chmod 700 /etc/amandapass
· Configuration
done on Windows client Uranium:
The
directory
getting backed up must be shared from Windows and must be
accessible by the Windows user zmanda with the password amanda.
accessible by the Windows user zmanda with the password amanda.
Set Backup Parameters
1. On
Quartz, as the amandabackup user,
create the Amanda configuration directory.
[root@quartz
etc]# su - amandabackup
-bash-3.00$ mkdir /etc/amanda/DailySet1
-bash-3.00$ mkdir /etc/amanda/DailySet1
2. Copy
the /var/lib/amanda/example/amanda.conf file to the /etc/amanda/DailySet1 directory. The amanda.conf file is
the most important file for configuring your Amanda setup.
-bash-3.00$
cp /var/lib/amanda/example/amanda.conf
/etc/amanda/DailySet1
3. The
sample amanda.conf distributed with
Amanda is over 700 lines long and is extensively commented. For more
information, search for amanda.conf on the Amanda
wiki. We will
focus on just a few lines and make
minimal modifications.
Open /etc/amanda/DailySet1/amanda.conf with
any text editor and edit it to suit your environment.
· The
following lines control some details specific to your organization and
to your
tape configuration.
org
"YourCompanyName" # your organization
name for reports
mailto "root@localhost" # space separated list of operators at your site
tpchanger "chg-disk" # the tape-changer glue script
tapedev "file://space/vtapes/DailySet1/slots" # the no-rewind tape device to be used
tapetype HARDDISK # use hard disk intead of tapes (vtape config)
mailto "root@localhost" # space separated list of operators at your site
tpchanger "chg-disk" # the tape-changer glue script
tapedev "file://space/vtapes/DailySet1/slots" # the no-rewind tape device to be used
tapetype HARDDISK # use hard disk intead of tapes (vtape config)
· We
add the following lines to specify the size of the virtual tapes:
define
tapetype HARDDISK {
length 100000 mbytes
}
length 100000 mbytes
}
· We
add the following lines to support the encrypted backup of /home/pavel on Iron:
define
dumptype encrypt-simple {
root-tar
comment "client simple symmetric encryption, dumped with tar"
encrypt client
compress fast
client_encrypt "/usr/sbin/amcryptsimple"
client_decrypt_option "-d"
}
root-tar
comment "client simple symmetric encryption, dumped with tar"
encrypt client
compress fast
client_encrypt "/usr/sbin/amcryptsimple"
client_decrypt_option "-d"
}
. Go to the “define
dumptype global”
section in the amanda.conf file and add the “auth
"bsdtcp"” line right before the last “}” bracket. This is done to
enable “BSDTCP” authentication.
# index
yes
# record no
# split_diskbuffer "/raid/amanda"
# fallback_splitsize 64m
auth "bsdtcp"
# record no
# split_diskbuffer "/raid/amanda"
# fallback_splitsize 64m
auth "bsdtcp"
4. As
the root user, create a cache directory to
use as a holding disk.
[root@quartz
~]# mkdir -p /dumps/amanda
[root@quartz ~]# chown amandabackup:disk /dumps/amanda
[root@quartz ~]# chmod 750 /dumps/amanda
[root@quartz ~]# chown amandabackup:disk /dumps/amanda
[root@quartz ~]# chmod 750 /dumps/amanda
5. Create
the virtual tapes. Dedicated directories are used as “virtual tapes”
called vtapes. You work with vtapes in the same
way that you work with physical tapes. Vtapes can even simulate tape
changers,
as you will see in our example.
For
security
reasons, limit access to the vtapes directory to the amandabackup user.
As the root user:
[root@quartz
~]# mkdir -p /space/vtapes
[root@quartz ~]# chown amandabackup:disk /space/vtapes
[root@quartz ~]# chmod 750 /space/vtapes
[root@quartz ~]# chown amandabackup:disk /space/vtapes
[root@quartz ~]# chmod 750 /space/vtapes
As the amandabackup user:
-bash-3.00$
touch /etc/amanda/DailySet1/tapelist
-bash-3.00$ mkdir -p /space/vtapes/DailySet1/slots
-bash-3.00$ cd /space/vtapes/DailySet1/slots
-bash-3.00$ for ((i=1; $i<=25; i++)); do mkdir slot$i;done
-bash-3.00$ ln -s slot1 data
-bash-3.00$ mkdir -p /space/vtapes/DailySet1/slots
-bash-3.00$ cd /space/vtapes/DailySet1/slots
-bash-3.00$ for ((i=1; $i<=25; i++)); do mkdir slot$i;done
-bash-3.00$ ln -s slot1 data
6. Test
the virtual tape setup.
-bash-3.00$
ammt -f
file:/space/vtapes/DailySet1/slots status
file:/space/vtapes/DailySet1/slots
status: ONLINE
file:/space/vtapes/DailySet1/slots
status: ONLINE
7. Just
as with physical tapes, the virtual tapes now need to be labeled.
(Please note
that the output below has been truncated.)
bash-3.00$
for ((i=1; $i<=9;i++)); do amlabel
DailySet1 DailySet1-0$i slot $i; done
changer: got exit: 0 str: 1 file://space/vtapes/DailySet1/slots
labeling tape in slot 1 (file://space/vtapes/DailySet1/slots):
rewinding, reading label, not an amanda tape (Read 0 bytes)
rewinding, writing label DailySet1-01, checking label, done.
...
changer: got exit: 0 str: 9 file://space/vtapes/DailySet1/slots
labeling tape in slot 9 (file://space/vtapes/DailySet1/slots):
rewinding, reading label, not an amanda tape (Read 0 bytes)
rewinding, writing label DailySet1-09, checking label, done.
changer: got exit: 0 str: 1 file://space/vtapes/DailySet1/slots
labeling tape in slot 1 (file://space/vtapes/DailySet1/slots):
rewinding, reading label, not an amanda tape (Read 0 bytes)
rewinding, writing label DailySet1-01, checking label, done.
...
changer: got exit: 0 str: 9 file://space/vtapes/DailySet1/slots
labeling tape in slot 9 (file://space/vtapes/DailySet1/slots):
rewinding, reading label, not an amanda tape (Read 0 bytes)
rewinding, writing label DailySet1-09, checking label, done.
-bash-3.00$
for ((i=10; $i<=25;i++)); do amlabel
DailySet1 DailySet1-$i slot $i; done
changer: got exit: 0 str: 10 file://space/vtapes/DailySet1/slots
labeling tape in slot 10 (file://space/vtapes/DailySet1/slots):
rewinding, reading label, not an amanda tape (Read 0 bytes)
rewinding, writing label DailySet1-10, checking label, done.
...
changer: got exit: 0 str: 25 file://space/vtapes/DailySet1/slots
labeling tape in slot 25 (file://space/vtapes/DailySet1/slots):
rewinding, reading label, not an amanda tape (Read 0 bytes)
rewinding, writing label DailySet1-25, checking label, done.
changer: got exit: 0 str: 10 file://space/vtapes/DailySet1/slots
labeling tape in slot 10 (file://space/vtapes/DailySet1/slots):
rewinding, reading label, not an amanda tape (Read 0 bytes)
rewinding, writing label DailySet1-10, checking label, done.
...
changer: got exit: 0 str: 25 file://space/vtapes/DailySet1/slots
labeling tape in slot 25 (file://space/vtapes/DailySet1/slots):
rewinding, reading label, not an amanda tape (Read 0 bytes)
rewinding, writing label DailySet1-25, checking label, done.
8. Now
we need to reset the virtual tape changer back to the first slot.
-bash-3.00$
amtape DailySet1 reset
changer: got exit: 0 str: 1
amtape: changer is reset, slot 1 is loaded.
changer: got exit: 0 str: 1
amtape: changer is reset, slot 1 is loaded.
9. Create
an /etc/amanda/DailySet1/disklist file in the Amanda configuration directory. The disklist contains the
fully
qualified backup client names, the directory or directories to be
backed up and
the dumptype.
copper.zmanda.com
/var/www/html comp-user-tar
iron.zmanda.com /home/pavel encrypt-simple
quartz.zmanda.com //uranium/MyDocuments comp-user-tar
iron.zmanda.com /home/pavel encrypt-simple
quartz.zmanda.com //uranium/MyDocuments comp-user-tar
10. As the
user amandabackup, append the following lines to the /var/lib/amanda/.amandahosts file to
allow the backup clients to connect back to the server when doing
restores.
Specify fully qualified domain names.
iron.zmanda.com
root amindexd amidxtaped
copper.zmanda.com root amindexd amidxtaped
quartz.zmanda.com root amindexd amidxtaped
quartz.zmanda.com amandabackup admump
copper.zmanda.com root amindexd amidxtaped
quartz.zmanda.com root amindexd amidxtaped
quartz.zmanda.com amandabackup admump
11. Create a
cron job that will execute
amdump and initiate your backups automatically. As the amandabackup user, run crontab
-e,and add the following line to run backups Monday through Friday
at 1am.
0 1 * *
1-5 /usr/sbin/amdump DailySet1
Verify Your Configuration
1. On
Quartz, as amandabackup, run the amcheck tool to verify that you can
successfully perform a backup.
-bash-3.00$
amcheck DailySet1
Amanda Tape Server Host Check
-----------------------------
Holding disk /dumps/amanda: 16714488 KB disk space available, using 16612088 KB
slot 1: read label `DailySet1-01', date `X'
NOTE: skipping tape-writable test
Tape DailySet1-01 label ok
NOTE: conf info dir /etc/amanda/DailySet1/curinfo does not exist
NOTE: it will be created on the next run.
NOTE: index dir /etc/amanda/DailySet1/index does not exist
NOTE: it will be created on the next run.
Server check took 4.259 seconds
Amanda Backup Client Hosts Check
--------------------------------
Client check: 3 hosts checked in 27.097 seconds, 0 problems found
(brought to you by Amanda 2.5.1p2)
Amanda Tape Server Host Check
-----------------------------
Holding disk /dumps/amanda: 16714488 KB disk space available, using 16612088 KB
slot 1: read label `DailySet1-01', date `X'
NOTE: skipping tape-writable test
Tape DailySet1-01 label ok
NOTE: conf info dir /etc/amanda/DailySet1/curinfo does not exist
NOTE: it will be created on the next run.
NOTE: index dir /etc/amanda/DailySet1/index does not exist
NOTE: it will be created on the next run.
Server check took 4.259 seconds
Amanda Backup Client Hosts Check
--------------------------------
Client check: 3 hosts checked in 27.097 seconds, 0 problems found
(brought to you by Amanda 2.5.1p2)
Run a Backup
1. On
Quartz, as amandabackup, run amdump to start the DailySet1 backup.
-bash-3.00$ amdump DailySet1
2. Amanda
will email a detailed status report from the amandabackup user
to you, the root user on Quartz.
From
amandabackup@quartz.zmanda.com Fri Jan 5 13:04:20 2007
Date: Fri, 5 Jan 2007 13:04:19 -0800
From: Amanda user
To: root@quartz.zmanda.com
Subject: YourCompanyName AMANDA MAIL REPORT FOR January 5, 2007
Date: Fri, 5 Jan 2007 13:04:19 -0800
From: Amanda user
To: root@quartz.zmanda.com
Subject: YourCompanyName AMANDA MAIL REPORT FOR January 5, 2007
These
dumps were to tape DailySet1-02.
The next tape Amanda expects to use is: a new tape.
The next new tape already labelled is: DailySet1-02.
The next tape Amanda expects to use is: a new tape.
The next new tape already labelled is: DailySet1-02.
STATISTICS:
Total Full Incr.
-------- -------- --------
Total Full Incr.
-------- -------- --------
Estimate
Time (hrs:min) 0:00
Run Time (hrs:min) 0:00
Dump Time (hrs:min) 0:00 0:00 0:00
Output Size (meg) 3.5 3.5 0.0
Original Size (meg) 11.8 11.8 0.0
Avg Compressed Size (%) 29.7 29.7 --
Filesystems Dumped 3 3 0
Avg Dump Rate (k/s) 292.8 292.8 --
Tape Time (hrs:min) 0:00 0:00 0:00
Tape Size (meg) 3.7 3.7 0.0
Tape Used (%) 0.0 0.0 0.0
Filesystems Taped 3 3 0
Chunks Taped 0 0 0
Avg Tp Write Rate (k/s) 8509.1 8509.1 --
Run Time (hrs:min) 0:00
Dump Time (hrs:min) 0:00 0:00 0:00
Output Size (meg) 3.5 3.5 0.0
Original Size (meg) 11.8 11.8 0.0
Avg Compressed Size (%) 29.7 29.7 --
Filesystems Dumped 3 3 0
Avg Dump Rate (k/s) 292.8 292.8 --
Tape Time (hrs:min) 0:00 0:00 0:00
Tape Size (meg) 3.7 3.7 0.0
Tape Used (%) 0.0 0.0 0.0
Filesystems Taped 3 3 0
Chunks Taped 0 0 0
Avg Tp Write Rate (k/s) 8509.1 8509.1 --
USAGE BY
TAPE:
Label Time Size % Nb Nc
DailySet1-02 0:00 3744K 0.0 3 0
DailySet1-02 0:00 3744K 0.0 3 0
NOTES:
planner: Forcing full dump of copper.zmanda.com:/var/www/html as directed.
planner: Forcing full dump of iron.zmanda.com:/home/pavel as directed.
planner: Forcing full dump of quartz.zmanda.com://uranium/MyDocuments as directed.
taper: tape DailySet1-02 kb 3744 fm 3 [OK]
planner: Forcing full dump of copper.zmanda.com:/var/www/html as directed.
planner: Forcing full dump of iron.zmanda.com:/home/pavel as directed.
planner: Forcing full dump of quartz.zmanda.com://uranium/MyDocuments as directed.
taper: tape DailySet1-02 kb 3744 fm 3 [OK]
DUMP
SUMMARY:
DUMPER STATS TAPER STATS
HOSTNAME DISK L ORIG-KB OUT-KB COMP% MMM:SS KB/s MMM:SS KB/s
-------------------------- ------------------------------------- -------------
copper.zmand -r/www/html 0 7640 2336 30.6 0:03 910.6 0:00 8680.7
iron.zmanda. /home/pavel 0 3530 1024 29.0 0:07 149.1 0:00 12486.1
quartz.zmand -yDocuments 0 960 384 40.0 0:03 101.0 0:00 4295.3
(brought to you by Amanda version 2.5.1p2)
DUMPER STATS TAPER STATS
HOSTNAME DISK L ORIG-KB OUT-KB COMP% MMM:SS KB/s MMM:SS KB/s
-------------------------- ------------------------------------- -------------
copper.zmand -r/www/html 0 7640 2336 30.6 0:03 910.6 0:00 8680.7
iron.zmanda. /home/pavel 0 3530 1024 29.0 0:07 149.1 0:00 12486.1
quartz.zmand -yDocuments 0 960 384 40.0 0:03 101.0 0:00 4295.3
(brought to you by Amanda version 2.5.1p2)
3. You
can also run the tool amadmin with a find argument for a quick summary of
what has been backed up.
-bash-3.00$
amadmin DailySet1 find
Scanning /dumps/amanda...
Scanning /dumps/amanda...
date host disk lv tape or file file part
status
2007-01-05 13:04:03 copper.zmanda.com /var/www/html 0 DailySet1-02 2 -- OK
2007-01-05 13:04:03 iron.zmanda.com /home/pavel 0 DailySet1-02 3 -- OK
2007-01-05 13:04:03 quartz.zmanda.com //uranium/MyDocuments 0 DailySet1-02 1 -- OK
2007-01-05 13:04:03 copper.zmanda.com /var/www/html 0 DailySet1-02 2 -- OK
2007-01-05 13:04:03 iron.zmanda.com /home/pavel 0 DailySet1-02 3 -- OK
2007-01-05 13:04:03 quartz.zmanda.com //uranium/MyDocuments 0 DailySet1-02 1 -- OK
In just about 15
minutes, we installed and configured a secure, heterogeneous network
backup, verified our configurations and ran a backup. We did it with
freely downloadable open source software that you can install from
binaries or compile for your unique needs. The pizza, which should be
getting delivered right about now, will be that much more enjoyable
with the clear conscience and peace of mind that comes with knowing
that your data is secure.
Recovery Based on feedback received on our forums we are adding a section that shows the ability to do a restore.
1. On Copper, as root, create the "/etc/amanda" directory.
copper:~ # mkdir /etc/amanda
copper:~ # chown amandabackup:disk /etc/amanda
copper:~ # chown amandabackup:disk /etc/amanda
2. As amandabackup, create a file "/etc/amanda/amanda-client.conf" and insert the lines below in to the file.
# amanda.conf - sample Amanda client configuration file.
#
# This file normally goes in /etc/amanda/amanda-client.conf.
#
conf "DailySet1" # your config name
index_server "quartz.zmanda.com" # your amindexd server
tape_server "quartz.zmanda.com" # your amidxtaped server
#tapedev "/dev/null" # your tape device
# auth - authentication scheme to use between server and client.
# Valid values are "bsd", "bsdudp", "bsdtcp" and "ssh".
# Default: [auth "bsdtcp"]
auth "bsdtcp"
# your ssh keys file if you use ssh auth
ssh_keys "/var/lib/amanda/.ssh/id_rsa_amrecover"
#
# This file normally goes in /etc/amanda/amanda-client.conf.
#
conf "DailySet1" # your config name
index_server "quartz.zmanda.com" # your amindexd server
tape_server "quartz.zmanda.com" # your amidxtaped server
#tapedev "/dev/null" # your tape device
# auth - authentication scheme to use between server and client.
# Valid values are "bsd", "bsdudp", "bsdtcp" and "ssh".
# Default: [auth "bsdtcp"]
auth "bsdtcp"
# your ssh keys file if you use ssh auth
ssh_keys "/var/lib/amanda/.ssh/id_rsa_amrecover"
3. As root run "amrecover" to initiate the data recovery process.
copper:/etc/amanda # amrecover
AMRECOVER Version 2.5.1p2. Contacting server on quartz.zmanda.com ...
220 quartz AMANDA index server (2.5.1p2) ready.
Setting restore date to today (2007-01-08)
200 Working date set to 2007-01-08.
200 Config set to DailySet1.
501 Host copper is not in your disklist.
Trying host copper.zmanda.com ...
200 Dump host set to copper.zmanda.com.
Use the setdisk command to choose dump disk to recover
amrecover>
AMRECOVER Version 2.5.1p2. Contacting server on quartz.zmanda.com ...
220 quartz AMANDA index server (2.5.1p2) ready.
Setting restore date to today (2007-01-08)
200 Working date set to 2007-01-08.
200 Config set to DailySet1.
501 Host copper is not in your disklist.
Trying host copper.zmanda.com ...
200 Dump host set to copper.zmanda.com.
Use the setdisk command to choose dump disk to recover
amrecover>
4. The list of commands below will demonstrate a recovery of a set of different files and directories to the "/tmp" directory.
amrecover> listdisk
200- List of disk for host copper.zmanda.com
201- /var/www/html
200 List of disk for host copper.zmanda.com
amrecover> setdisk /var/www/html
200 Disk set to /var/www/html.
amrecover> ls
2007-01-05-13-04-03 tar-1.15/
2007-01-05-13-04-03 .
amrecover> cd tar-1.15
/var/www/html/tar-1.15
amrecover> ls
2007-01-05-13-04-03 scripts/
2007-01-05-13-04-03 doc/
2007-01-05-13-04-03 configure
2007-01-05-13-04-03 config/
2007-01-05-13-04-03 COPYING
2007-01-05-13-04-03 AUTHORS
2007-01-05-13-04-03 ABOUT-NLS
amrecover> add scripts/
Added dir /tar-1.15/scripts/ at date 2007-01-05-13-04-03
amrecover> add configure
Added file /tar-1.15/configure
amrecover> add doc/
Added dir /tar-1.15/doc/ at date 2007-01-05-13-04-03
amrecover> lcd /tmp
amrecover> extract
Extracting files using tape drive chg-disk on host quartz.zmanda.com.
The following tapes are needed: DailySet1-02
Restoring files into directory /tmp
Continue [?/Y/n]? y
Extracting files using tape drive chg-disk on host quartz.zmanda.com.
Load tape DailySet1-02 now
Continue [?/Y/n/s/t]? y
./tar-1.15/doc/
./tar-1.15/scripts/
./tar-1.15/configure
./tar-1.15/doc/Makefile.am
./tar-1.15/doc/Makefile.in
./tar-1.15/doc/convtexi.pl
./tar-1.15/doc/fdl.texi
./tar-1.15/doc/freemanuals.texi
./tar-1.15/doc/getdate.texi
./tar-1.15/doc/header.texi
./tar-1.15/doc/stamp-vti
./tar-1.15/doc/tar.info
./tar-1.15/doc/tar.info-1
./tar-1.15/doc/tar.info-2
./tar-1.15/doc/tar.texi
./tar-1.15/doc/version.texi
./tar-1.15/scripts/Makefile.am
./tar-1.15/scripts/Makefile.in
./tar-1.15/scripts/backup-specs
./tar-1.15/scripts/backup.in
./tar-1.15/scripts/backup.sh.in
./tar-1.15/scripts/dump-remind.in
./tar-1.15/scripts/restore.in
amrecover> quit
200 Good bye.
200- List of disk for host copper.zmanda.com
201- /var/www/html
200 List of disk for host copper.zmanda.com
amrecover> setdisk /var/www/html
200 Disk set to /var/www/html.
amrecover> ls
2007-01-05-13-04-03 tar-1.15/
2007-01-05-13-04-03 .
amrecover> cd tar-1.15
/var/www/html/tar-1.15
amrecover> ls
2007-01-05-13-04-03 scripts/
2007-01-05-13-04-03 doc/
2007-01-05-13-04-03 configure
2007-01-05-13-04-03 config/
2007-01-05-13-04-03 COPYING
2007-01-05-13-04-03 AUTHORS
2007-01-05-13-04-03 ABOUT-NLS
amrecover> add scripts/
Added dir /tar-1.15/scripts/ at date 2007-01-05-13-04-03
amrecover> add configure
Added file /tar-1.15/configure
amrecover> add doc/
Added dir /tar-1.15/doc/ at date 2007-01-05-13-04-03
amrecover> lcd /tmp
amrecover> extract
Extracting files using tape drive chg-disk on host quartz.zmanda.com.
The following tapes are needed: DailySet1-02
Restoring files into directory /tmp
Continue [?/Y/n]? y
Extracting files using tape drive chg-disk on host quartz.zmanda.com.
Load tape DailySet1-02 now
Continue [?/Y/n/s/t]? y
./tar-1.15/doc/
./tar-1.15/scripts/
./tar-1.15/configure
./tar-1.15/doc/Makefile.am
./tar-1.15/doc/Makefile.in
./tar-1.15/doc/convtexi.pl
./tar-1.15/doc/fdl.texi
./tar-1.15/doc/freemanuals.texi
./tar-1.15/doc/getdate.texi
./tar-1.15/doc/header.texi
./tar-1.15/doc/stamp-vti
./tar-1.15/doc/tar.info
./tar-1.15/doc/tar.info-1
./tar-1.15/doc/tar.info-2
./tar-1.15/doc/tar.texi
./tar-1.15/doc/version.texi
./tar-1.15/scripts/Makefile.am
./tar-1.15/scripts/Makefile.in
./tar-1.15/scripts/backup-specs
./tar-1.15/scripts/backup.in
./tar-1.15/scripts/backup.sh.in
./tar-1.15/scripts/dump-remind.in
./tar-1.15/scripts/restore.in
amrecover> quit
200 Good bye.
5. We can now verify that the files have been recovered successfully by running run the following command.
copper:/ # tree /tmp/tar-1.15
/tmp/tar-1.15
|-- configure
|-- doc
| |-- Makefile.am
| |-- Makefile.in
| |-- convtexi.pl
| |-- fdl.texi
| |-- freemanuals.texi
| |-- getdate.texi
| |-- header.texi
| |-- stamp-vti
| |-- tar.info
| |-- tar.info-1
| |-- tar.info-2
| |-- tar.texi
| `-- version.texi
`-- scripts
|-- Makefile.am
|-- Makefile.in
|-- backup-specs
|-- backup.in
|-- backup.sh.in
|-- dump-remind.in
`-- restore.in
2 directories, 21 files
/tmp/tar-1.15
|-- configure
|-- doc
| |-- Makefile.am
| |-- Makefile.in
| |-- convtexi.pl
| |-- fdl.texi
| |-- freemanuals.texi
| |-- getdate.texi
| |-- header.texi
| |-- stamp-vti
| |-- tar.info
| |-- tar.info-1
| |-- tar.info-2
| |-- tar.texi
| `-- version.texi
`-- scripts
|-- Makefile.am
|-- Makefile.in
|-- backup-specs
|-- backup.in
|-- backup.sh.in
|-- dump-remind.in
`-- restore.in
2 directories, 21 files