• Answers
• BackupProposal

Backup Proposal

4 Feb 08

Summary

Over the last couple of months NICS has made some progress in developing an on-disk backup solution. We have utilized our experience with RAID based disk arrays and have extended the concept to backups. We have worked to keep the solution simple and flexible to incorporate future off-site backup capabilities, scaling up and ease of on-going maintenance. We are using built-in software utilities for a home-brew solution which can be easily modified to keep pace with changing needs. We have developed backup scripts and have created a setup to test the concept. The tests are going well. We have defined the scope of initial hardware needed and identified a vendor and part numbers. We have prices.

Layout

hosts (source - Mac, Linux or PC) connect to the proposed backup server (destination). Source is any NICS managed machine that we want backed up. Destination is where the backup data will be stored.

Equipment and Utilities

• Based on our current 3 RaidWeb RAID array experience, we have decided to utilize hardware based RAID in our backup solution and have decided to stick with the proven RaidWeb RAID array.

• The backup will be a disk array with a Solaris "head" machine and a UFS root filesystem.
• The backup disks will have a ZFS file system across multiple machines
• The backup system will be available for users with a NFS read-only mount over a secure shell (ssh) link and fiber channel.
• According to our current "Networker" software application, we need a minimum of 20 TB of data disk space.

• We currently have 6 TB available on our network. A RaidWeb RAID array of 16 drives yields the additional 14 TBs needed.

• Utilize the rsync software utility on Linux, on OS X and on PCs (known as DeltaCopy or Cygwin)

• Utilize ZFS SUN Solaris based filesystem for storage media.
• Utilize ssh, nfs, dsa key over fiber channel
• Utilize snapshot software utility to implement differential backups

Costs and Capacities

A RaidWeb RAID array with a 16 drive capability and 14 TB capacity currently costs $12K fully populated with drives. Assuming we want to keep a disk array chassis for 7 years, projected costs with estimated increased disk capacity usage may look like:

NOTE: Assumes 4 TB drives come out in 2010 and will cost what current 1 TB drives cost ($300/each)

Process

• Each host will periodically run the rsync utility to collect data which has changed from the last run and send it to the destination via a ssh link using dsa security.
• Once there, zfs will use the snapshot utility to create and store a 'snapshot' and make the backup read-only.

Risks

• We don't know the level detail we can achieve with Windows hosts.
• If we are handling 50+ TB of data, we may decide at some time in the future, to upgrade the hardware. Speed of backplane, fiber and headends may have to be upgraded This means changing out the chassis, fiber interfaces, fiber channel switches and head end machines.

Progress - Currently Testing

• We want to collect the data we want (changes since the last sync), send it securely, receive it and create a snapshot, store it as read-only in compressed format, make it available for a user over a secure NFS mount.
• We have created a zfs filesystem on a server
• We have developed beta versions of a rsync script which send the data to the destination and a zfs script which receives it and creates and stores it as a snapshot.
• We are actively exercising the prototype rsync setup using Linux, Solaris, OpenBSD, and FreeBSD hosts.