Java Solaris Communities About Sun How to Buy United States Worldwide

Solstice Backup and Storage Management - A Technical White Paper

January 1998

Contents:

Chapter 1: The Solstice Backup Product Family

Introduction

Data has become the enterprise asset. The value of data cannot be underestimated. Protecting and making it accessible to users is mandatory. Sounds straightforward? It's not. In the past several years, the protection and management of data and storage has developed from an administrative chore to become one of the most pressing and formidable challenges for system and network managers.

The primary factors attributing to this change are:

  • The growth in the amount of data stored in an enterprise.
  • The value of data is increasing.
  • The cost of managing on-line storage.
  • The cost to re-create lost data.
  • The lost productivity from system downtime and data inaccessibility.
  • Increasing requirements for data availability.

The overarching goal of network storage management is data accessibility. Data accessibility is accomplished by providing data protection, continuous availability, and high performance through various backup and storage management tasks and technologies.

FIGURE  1 Network Storage Management Needs

As the amount of network data grows faster than the storage space available, network managers are looking for ways to maximize their existing storage space without incurring large costs. To meet this need, a number of software vendors have introduced storage management products that reduce network storage and administration costs and improve data availability by automating and improving the manageability process of data backup, file storage, archival, migration, and retrieval.

Before proceeding to the discussion about specific product offerings, it is important to provide a definition of basic storage management elements:

  • Data Backup: Data backup is a process that stores redundant copies of files from hard disk to removable media, usually tapes. The redundant copies are "recovered" in case the original files are damaged or accidentally deleted. Backup is ordinarily scheduled to run on a daily basis. Backups are generally scheduled, controlled, and initiated by system administrators.

  • Data Archiving: Data archiving provides safe, reliable, organized long-term data storage. It is the process of taking a "snapshot" of a file or a series of related files as it resides on primary media (disk) at a given point in time. Once the snapshot is safely stored on removable media, the necessary files can optionally be deleted from the primary storage, allowing re-use of more expensive disk storage. Archiving is normally performed on files or data associated with specific projects. End users define specific archiving policies that match their application needs.

  • HSM: Hierarchical Storage Management (HSM) is a policy-driven data management strategy where data is moved or automatically migrated from one storage medium to another based on a set of policies. The policy that often controls the movement of data is "access rate," i.e. the longer the file has been inactive, the more likely it is to be migrated. After a file has been migrated from primary storage to secondary storage, it can later be staged to tertiary storage based on a set of policies.

The goals and characteristics of backup, archiving, and HSM can be summed up as follows:

  • The goal of backup is to protect data against accidental loss or damage. Backups should be reliable and efficient.
  • The goal of data archiving is to conserve on-line storage space. Reliability, safety, and storage on inexpensive media are the primary characteristics.
  • The goal of HSM is to decrease the overall cost of storage through better utilization of all the various storage mediums. Migration and de-migration should be automatic, transparent, and reliable.

As shown in Figure 2, the need for data backup, data archiving, and HSM are distinct and separate, and correspond to the amount of managed data on the network. It is very important for a site to ensure that a sound backup and archive strategy exists before an HSM solution is implemented.

FIGURE  2 Network Data and Storage Management

Sun is committed to simplifying the administration of network computing environments, protecting mission-critical data from loss, and managing the data stored on these networks in a consistent and cohesive fashion. Recognizing the immediate need from the user community for reliable and scalable data storage and management solutions, Sun offers the Solstice Backup product line1. This is Sun's strategic solution for addressing the mission-critical data protection and storage management needs of distributed, heterogeneous environments, particularly workgroups and departments. This paper outlines Sun product offerings for Solstice Backup, Solstice Archive, Solstice HSM, Solstice Storage Node, Solstice Database Module for Oracle, Informix, and SAP R/3 systems, and Solstice Backup Module for Microsoft SQL and Microsoft Exchange systems.

Chapter 2: Solstice Backup 5.0.1

Introduction

The information contained herein is intended for Sun system administrators and support specialists, sales representatives, and consultants as well as Sun customers. It provides important details regarding the features, benefits, and product stratification of the Solstice Backup server products. Sun has made available Solstice Backup Release 5.0.1 for Sun Solaris and SunOS environments. This white paper is an in-depth overview of the Solstice Backup and storage management solution.

Overview

Solstice Backup 5.0.1 storage management is an easy-to-use, enterprise-strength, highly scalable family of storage management applications for networks of all sizes. This release features greatly improved manageability, performance, and scalability, offering true lights-out data management for network storage management needs. Its superiority in manageability, performance, scalability, and ease of use is making Solstice Backup software one of the most trusted network data management products for IS departments worldwide.

Solstice Backup Model

The Solstice Backup architectural model relies on a Solstice Backup administration GUI that can "point to" and broker services for any Solstice Backup client on the network known to the Solstice Backup server. The Solstice Backup administration GUI can be displayed on any Solstice Backup machine in the network. The GUI is assembled on the fly as the connection is made to the machine being administered. In this way, the GUI reflects the specific features and functionality of the Solstice Backup software installed on the target machine. Solstice Backup shows the administrator only the options that are relevant for the target machine.

The Solstice Backup model introduced in this chapter relies on six components:

  • Server
  • Client(s)
  • Save sets (data)
  • Data-tracking databases
  • Pools of volumes
  • Configuration information

The Solstice Backup server can function as a client of itself, so in the simplest case, a Solstice Backup environment can be composed of a single machine that is both a client and the server (client software and server software reside on the same machine). A save set of data is the smallest unit of data on a machine that an administrator specifies for participation in a scheduled backup. An administrator-defined save set can include one or more files, directories, file systems, or "all files on the client machine." As data is backed up and written to volumes, Solstice Backup relies on two data-tracking databases: the client file index and the media database. The client file index is a browsable list of backed-up files, organized by Solstice Backup client. The index (which is stored internally as a database structure) receives a new entry each time a file is backed up. The index entry includes the time of the backup so that users can identify specific versions of a backed-up file. Entries remain in this browsable index for as long as the administrator wishes. Solstice Backup automatically removes entries from the client file index when the administrator-defined "browse policy" for the client's data expires. The media database receives an entry each time a save set is backed up and each time a storage volume is added to the Solstice Backup system. Entries remain in this database until they are manually removed by the administrator or the data on the volume is overwritten. Together, these three different kinds of tracking entries provide Solstice Backup with everything it needs to manage data.

The conceptual model includes storage volumes, not specific devices. Solstice Backup backup-and-recovery is independent of the device or media type. Conceptually, it is irrelevant whether the volumes that receive data are tapes, optical disk, or on-line disk. However, Solstice Backup only recognizes a volume within the context of a Solstice Backup pool of volumes. Administrators do not direct save sets either to a device or to a volume; they direct save sets to a named pool. In the simplest case, all Solstice Backup volumes would reside in a single pool named "Default:" and all save sets would be written to volumes in that pool.

Traditionally, software configuration information describes the application environment as specified by the administrator: What devices will Solstice Backup use? How many clients are defined? How many client connections can the server accept? In Solstice Backup, the configuration also describes the rules that the application enforces within the backup-and-recovery environment: What backup schedules are followed? How long is data protected before it is automatically recycled? Which users can recover data directly?

Server

The Solstice Backup server software includes programs and services that organize and execute Solstice Backup functions:

  • Solstice Backup operations require server coordination to define the save sets to be backed up; to read, package, and transfer client data back to the server; to create entries for Solstice Backup data-tracking structures; and to target the correct pool of volumes to receive the data.

  • Write operations require the server coordination to optimize performance by taking advantage of server parallelism; to manage writes between local and remote storage nodes; to multiplex (interleave) data savestreams to a single volume; to handle volume crossing when a single savestream cannot fit on a single volume; and to add tracking and synchronization information to the volumes.

  • Recover operations require the server to manage reads from the volumes; to optimize performance through server parallelism; to demultiplex (unweave and recreate a single savestream) data; and to coordinate the reproduction of the data on any one of the Solstice Backup supported client platforms.

  • Data lifecycle operations require that the server routinely compare the age and status of stored data with policies specified by the administrator, and take the action required to implement those policies.

  • Volume management operations require the server to locate volumes required by the write operations and to automatically mount, unmount, and label those volumes as needed; to inventory autochangers; and to clone and stage data from one volume to another as requested.

Each Solstice Backup server manages the data for its set of known clients. The administrator-defined service (or server) resource is the configuration information that defines the characteristics of the Solstice Backup server for this environment. Although Solstice Backup accommodates an unlimited number of clients, there is only one active server resource per environment.

Clients and Save Sets

The conceptual view of a Solstice Backup client is one or more save sets on a specific machine. When the Solstice Backup server backs itself up, it performs backup operations for its own save sets exactly as if it would back up the save sets located 3,000 miles away on the network.

Solstice Backup also enables administrators to define multiple Solstice Backup clients for a single machine. That is, a Solstice Backup environment can be composed of a single machine, but the configuration can define the environment as a Solstice Backup server that is administering multiple Solstice Backup client resources, where each client resource is a different collection of save sets residing on the client machine. For each combination of client machine and save sets that is a Solstice Backup client resource, an administrator defines a package of configuration information that includes its backup instructions and parameters.

Multiple client resources can share a common name in the administration GUI. The Solstice Backup interface does not enforce unique naming for Solstice Backup client resources, which are simply referred to as clients in the Solstice Backup GUI.

Data-Tracking Database

Taken together, the Solstice Backup client file index and media database entries maintain content and location information describing all backed-up data within the Solstice Backup environment. The entries' contents facilitate enforcement of data lifecycle policies defined by the administrator.

Users can graphically browse the Solstice Backup client file index to identify the filenames and available versions of immediately recoverable data. Version tracking and automatic file handling enable a Solstice Backup server to recreate client file systems to a point in time specifically requested by a user.

  • Writing entries to and performing lookups of the client file index
  • Writing entries to and performing lookups of the media database
  • Consistency checking of both client file index and media database
  • Compression of both client file index and media database space as needed
  • Database report preparation

When a unique situation arises, the Solstice Backup task-oriented GUI enables an administrator to supersede the automatic lifecycle policies and respond immediately as needed.

Pools of Volumes

When Solstice Backup begins operations, each volume is labeled and assigned to a named pool of volumes. Each Solstice Backup pool is associated with administrator-defined criteria that specify which save sets of data can be written onto the pool's volumes. A single pool can contain volumes of every supported media: on-line disk, optical disk, or tape. Solstice Backup directs save sets of data to the pool whose criteria most closely match the data characteristics. Thus, administrators direct data to pools, not to devices or to volumes.

When a save operation (such as a backup or an archive) begins, Solstice Backup searches for and mounts a volume from the appropriate pool of volumes. Solstice Backup writes save set data to the volume according to an industry-standardized format. This format meets the Solstice Backup requirement to store all information needed to reproduce data on any of the operating systems supported for Solstice Backup clients.

Any Solstice Backup volume can be read and its data recreated to a user-specified point in time by any other Solstice Backup server, regardless of server platform. Compatibility between volumes and future server releases is also guaranteed. Solstice Backup data from any Solstice Backup client, regardless of platform, can coexist on a single volume. Data from different save sets can be multiplexed to a single volume, or it can be split between two or more volumes.

Data can be recovered to the Solstice Backup server, to the client where the data originated, or to another specified client. Solstice Backup can also read a volume and recreate on the server new entries in the client file index and media database that describe the content of the volume.

Configuration Information

Solstice Backup configuration information is a database of records that administrators create and/or modify as they define a backup and storage management system. Administrators can create records that specify instructions to a fine level of detail, or they can rely on Solstice Backup software's many preconfigured records.

Each configuration database record represents a resource known to Solstice Backup. An abstract, conceptual definition of a Solstice Backup resource entity is difficult, but examples make the idea of a resource easier to understand. Examples of Solstice Backup configuration resource types follow:

  • Something that an administrator might need to specify (such as a backup schedule, a data lifecycle policy, or a client and its save sets).
  • Instructions that Solstice Backup might need to lookup (such as how to create a volume label, what kind of backup preprocessing to perform, or the conditions of a Solstice Backup license).
  • Something that Solstice Backup might need to locate (such as a device).

Most resources support multiple definitions (or records), for example multiple client resources, multiple schedules, and multiple devices, each of which is defined by a unique resource record. Solstice Backup comes with several preconfigured resource records that represent commonly used definitions. For example Solstice Backup comes with preconfigured resource records for weekly backup schedules and for data lifecycle periods such as a quarter, a month, and a year. Solstice Backup also comes with preconfigured resource records appropriate for a simple environment. For example, Solstice Backup includes a preconfigured record for a pool of volumes that accepts all save sets. The preconfigured pool is named "Default" and it is already associated with a set of preconfigured labels, also named "Default."

Solstice Backup Services: Names and Functions

At boot-time, Solstice Backup launches four services that run continuously on the server machine (and/or the storage node), even when the Solstice Backup application itself is idle.

The Solstice Backup server master service, nsrd, is responsible for starting the other three services. Descriptions of these four primary Solstice Backup services follow:

  • nsrd, the Solstice Backup server master service

    The nsrd service controls all Solstice Backup activity on the server, including authorizing and coordinating save and recover operations, monitoring sessions, and compiling server statistics. The nsrd service maintains the Solstice Backup configuration resource database and, during operation, initiates queries to the database on behalf of other services.

  • nsrindexd, the index management service

    The nsrindexd service manages the Solstice Backup server's on-line client file index, which by default stores an entry for every file or object that has been backed up. The client file index structure is browsable by users. Although administrators can choose to not store entries for backed-up data in the index, the primary function of the client file index is to answer the question: What files have been backed up?

  • nsrmmdbd, the media management database service

    The nsrmmdbd service manages the Solstice Backup server's media database, which tracks the location of the volumes within the environment and the location (position on the tape or disk) of backup data on each volume. The media database tracks the status of the data and the volumes, but its primary function is to answer the questions: What data is available for recovery, and where is it?

  • nsrmmd, the media management service

    The nsrmmd services provide device support including labeling; multiplexing writes (interleaving data) to media during backups; demultiplexing reads (unweaving the data) from media during recovery; and generating requests for volume mounting and unmounting when appropriate.

In Solstice Backup 5.0.1, an nsrmmd service is created for each enabled device known to Solstice Backup. The nsrmmd service runs on the machine where the device is located. That is, to support remote devices, nsrmmd services run on the storage node machine where the remote devices reside. The technology underlying storage node support theoretically permits all Solstice Backup 4.2 (and newer) clients to use Solstice Backup server storage nodes.

Figure 1 illustrates the simple Solstice Backup model and the services running on the Solstice Backup server and on a storage node machine at start up.

FIGURE  1 Solstice Backup Services Launched at Start Up

Sun's Product Offerings

Many customers are seeing a dramatic growth in the size and complexity of their computing environment. The Solstice Backup modular product structure accommodates varying customer needs. The product can be expanded to meet the needs of growing environments, thus protecting customers investment in Solstice Backup. Today, three server offering are available from Sun.

Solstice Backup Server Edition

Re-centralization is occurring among many of today's high-end file and database servers. Many customers are consolidating several servers into one larger server for cost or administration savings. Designed to address the need for single system backup, Solstice Backup Server Edition software brings the high-performance, automated, unattended parallel backup and restore capability of Solstice Backup to standalone servers. For example, Solstice Backup Server Edition will perform a simultaneous parallel backup from 32 different (non-RAID based) hard disks storing the data on 16 different tape drives concurrently. When combined with one of the Solstice Backup Database and Application Modules, customers will be able to back up and restore up to 32 tablespaces or data files simultaneously. Solstice Backup Server Edition is pre-configured to support jukebox and advanced data management options, and can be upgraded to Solstice Backup Network Edition software.

Solstice Backup Network Edition

The Solstice Backup Network Edition product is an enterprise-strength application that delivers network storage backup for distributed network clients. It is highly scalable with superior manageability, availability, and performance. Any system with a local hard disk, ranging from a single PC to a multi-processor Solaris server, can act as a client to Solstice Backup Network Edition. Solstice Backup Network Edition supports options that add client connections, expand client platform coverage, and deliver advanced data management services such as archiving, HSM, SNMP manageability, and Solstice Backup Database and Application Module support. It is pre-configured with ten client connections, and includes client software to protect all Sun-platform clients. It also supports jukebox and advanced data management options.

Solstice Backup Network Edition can be upgraded to Solstice Storage Management Suite software, which enables centralized and automatic management of multiple server and desktop systems from a single location.

Solstice Storage Management Suite

With the growth in enterprise-wide storage, network managers are looking for ways to allow them to reduce operational costs and administrative intervention. Solstice Storage Management Suite is a centrally-controlled, integrated suite of storage management applications that consolidates backup, Hierarchical Storage Management (HSM), and archiving under a consistent framework. It enables network managers to create a centralized storage center for the distributed network to deliver comprehensive storage management across enterprise networks.

The Solstice Storage Management Suite solution is available at a special price of up to 20% off the retail price of its individual components. Solstice Storage Management Suite includes:

  • Solstice Backup Network Edition (includes 10 client connections)
  • Archive Module
  • HSM Module
  • SNMP Module
  • Jukebox Software Module (1-16 slots)

Refer to the Solstice Backup Product Structure white paper for more details about available options and licensing technology and procedures.

New Features

The Solstice Backup product has powerful features that enable system administrators and IS personnel to continue to implement a state-of-the-art, consistent network storage management strategy for their organizations. These features encompass all areas of the product and are described next.

The key new Solstice Backup 5.0.1 features include:

  • Improved manageability, enabling centralized management of all storage operations.

  • Improved scalability by enabling the distribution of backup performance across the network to secondary servers while maintaining central control at the primary server.

  • Improved performance by supporting storage nodes, or storage devices attached to remote machines.

  • Improved data availability by enabling the creation of clone sets to off-site locations, performing failover backups on secondary servers, and requiring only backup server indices to recover data.

  • Enhanced high availability by supporting the ability of clients to back up to the first available device in their priority list.

  • Large file support -- provides backup, browse, and recover services for the 64-bit file systems supported on Solaris 2.6.

  • Greater flexibility and increased performance, with reduced backup time -- administrators can now choose to perform backup and recover activities from hard disk instead of tape drives.

  • Better scalability with support for multiple index files per client.

  • Expanded database and application backup support that now includes Microsoft SQL and Microsoft Exchange systems.

  • Support for Solaris 2.6.

Highlights

Solstice Backup 5.0.1 extends the benefits of Solstice Backup 4.2.x in the following areas:

Improved Performance

Solstice Backup storage management has been fine tuned to provide better performance -- a crucial need for mission-critical environments. With the Solstice Backup flexible client/server model, performance improvements can be made on the client or server side without affecting compatibility. For example, in Solstice Backup 5.0.1, devices that are not locally attached to the primary backup server can be utilized, resulting in greater performance. Additionally, Solstice Backup 5.0.1 provides the ability to backup directly to magnetic disk devices. This capability enables faster backup, data staging, and multi-level HSM activities. The benefits that result from improved performance in Solstice Backup 5.0.1 are increased reliability (since the probability that backup tasks will be completed is higher), reduced investment in tape devices (since existing storage resources may be utilized), and the ability to optimize backup performance at a lower cost.

Scalable Architecture

Today, it is common for a Solstice Backup server to provide data management services to hundreds of machines, protecting millions of files and hundreds of gigabytes of network data. As the installed base grows, it is expected that typical Solstice Backup servers could protect thousands of machines on the network. Solstice Backup offers a truly flexible architecture that can adapt to diverse organizational, administrative, and computing environments. Solstice Backup takes full advantage of new hardware resources (such as disks, clients, storage technology, or network backbones). It also provides flexible administrative tools, enabling customers to manage multiple Solstice Backup servers in a centralized or decentralized fashion.

The architectural direction for the Solstice Backup technology is to become infinitely scalable. One of the key problems that is addressed by the Solstice Backup 5.0.1 software is the network overload that occurs when client system data is backed up over a network to a single Solstice Backup server. Solstice Backup 5.0.1 now supports secondary backup servers and the backup of data to remote (network) devices not locally attached to the primary Solstice Backup server. Solstice Backup performance enhancements will continue to be a key theme for any future releases.

Improved Data Availability

Solstice Backup 5.0.1 helps to ensure the accessibility of data to end users and applications, thus reducing costs to the business resulting from down-time. The ability to create clone sets over the network to an off-site location enables the recovery of data in the event of catastrophic failure at the central backup server location. Additionally, Solstice Backup 5.0.1 supports the ability to automatically "fail-over" or move backup activities to a secondary server(s) in case the primary server fails. This occurs transparently (in the background), with no operator intervention required. In the event of a disaster, you can recover a system faster because only the backup server indices are needed to recover data.

Scalability Improvements

Solstice Backup 5.0.1 protects your storage management investment and improves productivity by providing a truly scalable solution to storage management problems. You can now distribute backup activities across the network to secondary servers while maintaining central control at the primary server. This enables you to easily extend your backup capabilities while keeping system and network administration costs in check. Solstice Backup 5.0.1 now supports files and indices greater than 2 Gbytes, providing protection for environments where large files are commonplace.

Manageability Improvements

Solstice Backup 5.0.1 reduces total cost of ownership costs and leverages existing system and network administration resources by enabling greater centralization and control of backup activities. All types of storage operations, including local and remote backup tasks, can be centrally managed using the Solstice Backup 5.0.1 software. You can save time and achieve conformity across your network by creating backup policy for a hierarchy of servers. Storage resources are used more efficiently and storage management activities can be automated by creating policy to move data from type of storage to another.

Storage Node Support

The Solstice Backup 5.0.1 Network Edition product now includes optional software that enables you to designate systems on the network that have attached storage devices to act as storage nodes of the backup server. Storage node machines manage the media that contains backed up data, while the information required to administer the clients and to track and recover the data is managed by the Solstice Backup server. In the Solstice Backup administration program, you affiliate Solstice Backup clients with a storage node system for backup and recovery requests.

The distribution of media management tasks to other systems on the network reduces the load placed on the controlling backup server and provides a means to globally manage remote storage management tasks across a distributed enterprise network from a central location.

Backup supports a variety of media types, including 4mm, 8mm, optical disk, digital linear tape (DLT), and disk file. It also supports a wide variety of devices attached to a Solstice Backup server or a designated storage node, either as a standalone device or in an autochanger or silo tape library. The list of supported media and devices continually grows. To obtain the latest list of supported devices, use the 24-hour FaxWorker interactive Fax server at (415) 812-6156. Follow the instructions supplied by FaxWorker, and request Document 1905, the Backup Compatibility Guide. This information is also available on the Web using the Legato Document Library (http://www.legato.com/).

Solstice Backup is HSM Ready

With Solstice Backup 5.0.1 storage management, Sun offers the Solstice Backup Hierarchical Storage Management (HSM) application. HSM optimizes your local storage resources by dynamically and transparently migrating less frequently referenced files to near-line storage. The more expensive local disk space is reclaimed for housing new data or data with higher performance access requirements.

Solstice Backup lets you use optical and tape libraries as well as primary media (magnetic disk) for storage of migrated files. Solstice Backup HSM is available as an add-on module for both Solstice Backup Server Edition and Network Edition customers, or as part of the fully integrated Solstice Storage Management Suite storage management environment combining Solstice Backup, HSM, and Archive. For more details about HSM, refer to "Solstice HSM" on page 45.

Simplified Software Installation

During product installation, all standard and optional Solstice Backup features are installed on the hard disk. Using the Solstice Backup innovative enabler technology, only the components that the customer purchased are accessible. When new options are purchased, the customer receives an "enabler certificate" that unlocks the new features without having to re-install new software or take down the server. For example, if the user wishes to upgrade the server to provide HSM services, all they need to do is enable the server with an appropriate enabler code -- without adding any extra software to the server or client. This technology also enables the customer to evaluate new functionality for up to 45 days without charge simply by using an evaluation enabler code. Evaluation codes are available from Sun for all optional components of Solstice Backup.

Extensive Device Support

Solstice Backup 5.0.1 storage management includes support for the following tape devices, Jukeboxes, Silos, and high-speed devices:

  • HP SureStore 5000
  • HP C1537A
  • Sony SDT9000
  • Exabyte 8900 Mammoth
  • IBM Mammoth
  • IBM 3590 B-01
  • Ampex DST-310
  • Sony DTF
  • Sony TSL-7000
  • Exabyte EXB-220 CHS
  • IBM 7331
  • ADIC Scalar 218
  • Breece Hill Q7
  • Phillips LMS Blackjack 21
  • HP 40fx
  • IBM 3494
  • StorageTek Wolfcreek
  • StorageTek Powderhorn
  • EMASS AML

For more details regarding devices supported by Solstice Backup, refer to Legato's Compatibility Guide on the Web at http://www.legato.com/.

Solstice Backup Features and Benefits

Table 1 summarizes the features and benefits of the Solstice Backup 5.0.1 product.

TABLE  1   Solstice Backup Features/Benefits 
Features Function Benefit
Heterogeneous platform support for backup and storage management service  Backup and data management of a wide array of systems and database platforms.  One, consistent, centralized enterprise-wide data protection and storage management solution. 
Support for all leading storage devices  Supports a wide variety of popular tape and optical devices (autoloaders and libraries), including Silos.  Supports automated, unattended operations, resulting in reduced cost of administration. 
Backup, recovery, and archival services for use by the end-user  End users can operate on their own files without relying on system administration.  Empowers the end user and reduces the overall cost of administration. 
Support for client backup parallelism  Provides the ability to back up as many as 32 different sessions in parallel to a server.  Improves client backup performance, thus reducing backup time. 
Backup device support for concurrent operations  Supports the backup of data to up to 16 devices concurrently.  Results in more backup throughput, thus reducing backup time. 
Storage nodes and remote backup capabilities  Back up data to a distributed secondary server(s) and device(s) while maintaining central control/management.  Ensures that client machines get backed up to the nearest server/device, allowing distribution of network backup load and improving performance and reliability. 
Data staging  Ability to back up data to magnetic disks and later move data off to tape.  Backup/restore performance is improved and cost-effective use of storage resource is enhanced. 
Archival services  Ability to take a `snapshot' of a set of data and safely move to secondary storage and optionally delete from on-line storage.  Long-term reliable data protection and conservation of disk space. 
Hierarchical Storage Management  Multi-level, policy-driven, automatic and transparent migration and recall of data.  Optimize utilization and performance of system, storage, and administrative services. 

Integrated database and application backup 

Provides simplified administration, scheduling capabilities, and automated media management for Oracle, Informix, and SAP R/3 systems as well as Microsoft SQL and Exchange Servers.  Consistent, reliable, high-performance data protection as well as automated unattended operations. Reduces cost of administration. 
Network management integration  Ability to send event notification and status to network management console for monitoring the storage management network.  Centralized network management for easier administration of complex, distributed environment; higher data availability. 

Solstice Backup and Database Module Evaluation

Sun is pleased to offer Solstice Backup 5.0.1 evaluation products on CD-ROM. This evaluation copy provides customers with the entire suite of Solstice Backup storage management applications. For 45 days, customers can evaluate server extensions such as Autoloader or SNMP software modules, and storage management applications such as Solstice Backup, Archive, and HSM. This CD-ROM contains both Solaris SPARC and Solaris Intel server and client software. It also contains the complete product documentation on-line in html format, along with Sun AnswerBook versions of the manuals as well. If a user wishes to purchase the Solstice Backup 5.0.1 product, all they need to do is enable the evaluation server with an appropriate enabler code -- without adding any extra software.

Sun also offers a Solstice Backup Database Module Evaluation product. This CD-ROM contains Solstice Backup modules for Oracle, Informix, SAP R/3. Solstice Backup modules for Microsoft SQL and Microsoft Exchange systems are available on a separate evaluation CD.

Release Overview and Compatibility

Server Platforms

Solstice Backup 5.0.1 is available on the following platforms:

  • Sun SPARC systems running Solaris 2.4, 2.5, 2.5.1, or 2.6 as a client or server, or SunOS 4.1.4 or Solaris 2.3 as a client.

Solstice Backup HSM support is available for the following platforms:

  • Sun SPARC systems running Solaris 2.4, 2.5, 2.5.1, or 2.6 as a client or server, or SunOS 4.1.4 as a client.

Client Platforms

Solstice Backup software comes with client support for the server's operating system. Solstice Backup supports a wide range of platforms through ClientPak products, including:

  • IBM AIX
  • SunOS/Solaris
  • HP-UX
  • SGI IRIX
  • SCO UNIX -- OpenServer
  • UnixWare
  • NetWare
  • OS/2 Warp
  • Microsoft Windows
  • Windows NT
  • MacOS

While older versions of these clients will operate with Solstice Backup 5.0.1 software, installation of the most current versions of client software is required to achieve the best server-to-client performance.

Product Availability

Solstice Backup 5.0.1 for both Solaris SPARC and Solaris Intel platforms is now available through all Sun Channels. Sun is shipping the Solstice Backup version that is equivalent to Legato's most current version of Legato NetWorker® software.

Compatibility

Solstice Backup 5.0.1 software is compatible with other versions of Solstice Backup in all respects, including the version of Legato NetWorker available from other OEM partners of Legato. Data backed up with an earlier version of Solstice Backup server is recoverable with Solstice Backup 5.0.1. All existing Legato NetWorker ClientPaks are also compatible with this release.

Chapter 3: Solstice Backup Database and Application Modules

Introduction

As increasing numbers of organizations implement relational databases and other applications for business automation and mission-critical downsizing projects, the need for efficient, automated storage management of database and business applications is highlighted. The Solstice Backup family offers a solution that provides fast, on-line backup and restore capabilities for key databases and applications.

The Database Backup Problem

The size of relational databases on UNIX® networks has been growing at a phenomenal rate. According to Gartner Group, the average database will grow 1000% between 1995 and 2000; an average of 200% per year. Typically, RDBMS applications range from 10 to 100 Gbytes in size. Two major trends are driving the growth of the UNIX RDBMS market: "downsizing" and "upsizing." In the first case, UNIX relational databases are being used in mission-critical downsizing applications to support business operations that were once exclusive to the mainframe. For example, applications such as On-Line Transaction Processing (OLTP) and Executive Information Systems (EIS) are moving from mainframes to UNIX environments where users can exploit the high performance and cost competitiveness of open systems. In the second case, corporations are also deploying database applications for upsizing projects to re-engineer business operations for greater efficiency and cost savings.

Unfortunately, the advances in relational database technology to support large, distributed UNIX databases have not been matched by corresponding improvements in the system management tools that are essential to administer and back up such huge amounts of data. Database vendors have developed basic backup tools, but they do not scale well to the requirements of large databases, heterogeneous networks, or distributed data center operations.

Some of the most prevalent challenges associated with backing up UNIX relational databases include:

  • How to complete backups for large and growing databases in the allotted time.
  • How to provide adequate backup protection for high-availability databases without disrupting users.
  • How to provide absolute assurance of database integrity.
  • How to include backup operations for databases as part of a consolidated, enterprise-wide strategy for storage management as well as leveraging the corporations investment in staffing, procedures, and hardware.

Why Existing Backup Methods Fall Short

Currently, the tools most commonly available to database administrators for backup are based on standard UNIX utilities such as dump and tar. In typical usage, the database administrator writes shell scripts based on these utilities to perform database backups. However, as databases grow in size and complexity, the task of protecting large amounts of data rapidly outpaces the performance of these tools. Database administrators have come to understand some of the following inherent limitations of these tools for database backup protection.

Difficult Administration

  • Requires intimate knowledge of database structures and physical locations of database objects. Scripts grow quickly in size and complexity, and must be constantly reworked to account for any change in system configuration.
  • Difficult to train new operators due to the complexity of the scripts; database recovery requires a high level of UNIX and database expertise.
  • Difficult to implement and use for scheduling network-wide backups to a central site.

Slow Performance

  • Most utilities can only back up a single disk partition to a single tape drive; they fail to take advantage of optimal device speed or data density.
  • "Over-the-wire" performance is even slower.

Poor Database Availability

  • For on-line backups, lack of coordination between backup for data files and database transaction logs can produce inconsistent snapshots of locked files; this puts database integrity at risk for restore operations.
  • For off-line backups, as the database grows in size, lack of performance means backup will often fail to complete in the allotted time; availability requirements often limit database size.

Poor Media Management

  • No support for tape or optical autoloaders; this causes unnecessary administrative burden for daily backup maintenance.
  • No support for automatic media spanning, which means constant operator interaction is required to load and unload tapes, virtually eliminating the possibility of automation.
  • No support for automatic media labeling, media tracking, or media rotation.
  • No contingency to recover past bad spots on the media.

Requirements for Database Backup and Recovery

The growth of relational database applications on client/server UNIX systems will continue to increase dramatically, taxing the already heavy burden imposed on administrators. Continued reliance on unstructured and arcane techniques is unacceptable, especially for mission-critical mainframe replacement applications, where even a small amount of unscheduled downtime can be extremely costly to the organization.

To address the needs of system and database administrators for database backup and recovery, the following fundamental set of storage management features are required:

  • Consistent, reliable data protection.
  • High-performance operation with maximum database availability.
  • Simplified backup/recovery administration.
  • Scheduled, unattended on-line and off-line backups.
  • Automated media management.
  • Integration with other file-system backup requirements and heterogeneous networks.

The Backup and Recovery Solution for Key Databases and Applications

Ideally, the solution for relational database protection would comprise two optimized storage management services:

  • A database knowledgeable "front-end" to manage the details required to `quiet' the database in preparation for on-line backup and restore operations while maintaining database integrity.
  • A robust "back-end" service to handle the complexities of high-performance operation, media management, and scheduling in a consistent fashion across all systems on the network.

This is the architectural approach taken in developing Sun's backup and restore solutions for:

  • Oracle7 databases
  • Informix databases
  • SAP R/3 systems using Oracle
  • Microsoft SQL
  • Microsoft Exchange

Solstice Backup is a high-capacity, easy-to-use, network data storage management solution that provides backup and recovery for heterogeneous networks of servers and clients. Solstice Backup client/server architecture enables it to provide easy interoperability for virtually every network operating system, desktop operating system, and storage device on the market. But the Solstice Backup key benefits are its superior data protection and powerful, automated approach to simplifying the management of network backup and recovery.

A Cooperative Solution

Sun has teamed up with key database and application vendors to provide a reliable, high-performance backup and recovery solution for their respective platforms. The database and application vendors provide front-end services through "database-aware" utilities that manage the communication with the database. Sun has developed the Solstice Backup Database and Application Modules that supply the connection between the DBMS front-end services and the robust set of back-end services provided by Solstice Backup.

Solstice Backup software, used in conjunction with the Solstice Backup Database Modules, provides fast on-line backup and restore capability for databases with minimal user disruption.

FIGURE  1 Database Backup and Restore with Solstice Backup

Fulfilling the Requirements for Database Backup and Recovery

Consistent, Reliable Data Protection

Solstice Backup software works with the key DBMS backup utilities to provide comprehensive protection for large databases. Solstice Backup extends this protection with built-in features for consistent enforcement of backup policies, automatic data verification during backup, automatic notification of operation completion, and comprehensive media management.

Unmatched Performance

Solstice Backup storage management and the respective DBMS utilities work together to provide optimum performance during backup and recovery operations. The Solstice Backup/DBMS environment achieves superior performance for backup and restore operations with its ability to stream data from multiple tablespaces and multiple data servers in parallel to a single device. With parallelism, Solstice Backup eliminates a common bottleneck resulting from tape drives starting and stopping by allowing the tape device to stream data at its maximum speed. The respective DBMS backup utilities extend this parallel performance by offering concurrent read/write access for tablespaces or data files spread across multiple disks spindles. Solstice Backup can provide even faster performance with its ability to drive multiple tape devices or autoloaders concurrently.

FIGURE  2 Solstice Backup Can Back Up Multiple Tablespaces and Data Servers Simultaneously.

The parallelism built into the Solstice Backup architecture, together with the ability to drive data storage devices at their optimal speed, provides the fastest backup speeds possible. In addition, Solstice Backup throughput scales efficiently as processor boards, disk spindle, and storage devices are added to the system.

High Availability, On-line Operation

DBMS and application vendors supply backup utilities that enable on-line or off-line backups of databases and applications. Off-line backups include all database files. On-line backups can include all database files or a subset of those files. Subsets may comprise datafiles, tablespaces, control files, or parameter files. On-line partial saves can be used to minimize the backup time required, while still keeping restore and recovery times within acceptable limits. Fast, parallelized streaming of data files provides rapid job completion and, thus, minimal user disruption for backup operations.

The Solstice Backup/DBMS environment also supports on-line operation for partial restores of tablespaces and data files. However for a complete restore operation, the database would need to be taken off-line. Point-in-time restores enable the administrator to recover the database to a specified prior state.

Automated, Unattended Operations

The Solstice Backup Jukebox Module option provides fully automated, unattended "lights-out" backup and recovery by dramatically reducing media handling. The Solstice Backup media manager automatically labels and tracks valid tapes, eliminating the possibility of accidental erasure.

Using media pools, the network or database administrator can segregate database backups from other backups on a separate set of media, or send large database backups to the fastest media devices available.

FIGURE  3 Using Media Pools, Database Backup Data Can Be Separated From Other Types of Backup Data.

Simple, Enterprise-wide Backup Administration

With Solstice Backup software, a system administrator can schedule a database backup from any node on the network. A Motif graphical user interface allows for easy, intuitive navigation of the Solstice Backup feature set. Solstice Backup simplifies backup and recovery and reduces the administrative burden by automating and centralizing backup operations. In addition, the Solstice Backup client/server architecture provides a consolidated storage management solution for heterogeneous networks of servers and desktop systems.

FIGURE  4 Databases Can Be Administered From Any Solstice Backup Node on the Network.

Summary

Together, Sun and key database and application vendors have teamed up to provide a robust and efficient system to protect mission-critical databases in the corporate enterprise. The Sun DBMS and application vendor solutions provide functionally rich, flexible, high-integrity, and high-performance data protection systems for Oracle, Informix, and SAP R/3 databases, and Microsoft SQL and Microsoft Exchange systems. Sun's solutions can fit into the storage management and data protection strategy for the entire enterprise.

Statement of Direction

Sun's direction is to bring to market powerful data management products that offer robust data protection services for distributed LAN installations, including all popular UNIX, Windows NT, and NetWare environments. Sun recognizes that database backup is one of the most strategic data management requirements facing IS managers today. As part of the company's strategy, Sun intends to provide robust, platform-independent, storage management services for all of the most popular relational database products on the widest possible range of Sun and UNIX vendor platforms.

Sun will continue to work closely with other leading database vendors to develop an API-level interface to Solstice Backup for on-line database backup and recovery services. A module for Sybase is planned for development.

Configuration/Licensing

The Solstice Backup Database and Application Module(s) are add-on options for Solstice Backup Server Edition and Solstice Backup Network Edition. The backup/storage management server runs the Solstice Backup administrative software with a locally attached backup device. The system(s) running the database server software and the Solstice Backup Database or Application Module software is/are configured as clients to the backup server. The storage management server software and the database server software may run on the same or separate systems. Each database or application backup client must be separately licensed with Solstice Backup Database or Application Module software.

Specifications

Solstice Backup Requirements

Solstice Backup Database Module for Oracle 2.0:

  • Sun SPARC system running Solaris 2.3, 2.4, or 2.5.1.
  • Solstice Backup 4.2 or higher. (NOTE: In order to install Solstice Backup Database Module for Oracle on an Solstice Backup 4.1.2 system, a patch is required. See the Solstice Backup Database Module for Oracle release notes for details)

Requires Solstice Backup Network or Server Edition.

Oracle Requirements:

  • Target database: Oracle release 7.1 or higher.
  • Oracle's Enterprise Backup Utility Catalog Database: one instance of Oracle 7.1 or higher (20-30 Mbytes) to hold configuration information.

The Enterprise Backup Utility provides a database-aware mechanism to back up and restore Oracle data files, control files, and redo logs, while maintaining database security and referential integrity. Oracle's Enterprise Backup Utility manages backup/restore operations for Oracle7 databases on raw devices, file systems, or logical volume managers. The utility maintains a backup catalog in an external database to track configuration information and detailed backup histories for all target databases. When the data is being restored, the Enterprise Backup Utility checks data block addresses to ensure there are no gaps. It also verifies that data block numbers correspond to actual data files.

Solstice Backup Database Module for Informix 1.0:

  • Sun SPARC system running Solaris 2.3, 2.4, 2.5.1, or 2.6.
  • Solstice Backup 4.2 or higher.

Requires Solstice Backup Network or Server Edition.

Informix Requirements:

  • Target database: Informix Online Dynamic Server, v 7.21 or higher. This release includes the ON-Bar (OnLine Backup and Restore) Utility required for the Solstice Backup/Informix solution.

Informix ON-Bar Front-End Services provide off-line and on-line database management and log management as well as full and partial backups and restores. On-line partial saves are available to the dbspace or datafile level. Dbspaces, blobspaces, or datafiles can be multiplexed to maximize performance. Point-in-time restores are also supported. Damaged files can be quickly restored from media using ON-Bar. Recoveries are performed using standard Informix procedures.

Solstice Backup Database Module for SAP R/3 Systems v1.1:

  • Sun SPARC system running Solaris 2.3, 2.4, 2.5.1, or 2.6.
  • Solstice Backup 4.2 or higher.

Requires Solstice Backup Network or Server Edition.

SAP R/3 Requirements:

  • Target application/database: SAP R/3 v2.2 and 3.0 running Oracle. This application server includes the sapdba utility required for this backup solution.

Solstice Backup Module for SAP is a high-performance implementation of SAP's Backint specification that integrates SAP backup utilities with Solstice Backup solutions. The Solstice Backup Module for SAP R/3 systems has been SAP certified. These solutions provide reliable protection of local and distributed R/3 data, including tablespaces, redo logs, control file, and log files. The SAP backup utilities enable backup of multiple tablespaces and servers in parallel. The Solstice Backup solution integrates tightly with SAP environments to deliver added functionality while leveraging existing procedures. Users can perform backup and restore of SAP data to a Solstice Backup server using SAP's familiar sapdba utility. In addition, SAP-specific events can be sent via pager or e-mail using Solstice Backup's automated notification system.

Solstice Backup Module for Microsoft SQL Server:

  • Sun SPARC system running Solaris 2.3, 2.4, 2.5.1, or 2.6.
  • Solstice Backup 5.0.1 or higher.

Requires Solstice Backup Network or Server Edition.

Microsoft SQL Requirements

  • Target application/database: Microsoft SQL Server 6.0 and 6.5

The Solstice Backup Module for Microsoft SQL Server delivers on-line data protection for SQL Server databases. The Solstice Backup Module for Microsoft SQL Server is an implementation of the standard API facilities provided by SQL Server, providing a high-performance connection between Solstice Backup and SQL Server. The Solstice Backup Module for Microsoft SQL Server integrates backup and restore services for SQL Server databases, transaction logs, and control files into Solstice Backup's comprehensive, enterprise-strength storage management solution.

Solstice Backup Module for Microsoft Exchange Server:

  • Sun SPARC system running Solaris 2.3, 2.4, 2.5.1, or 2.6.
  • Solstice Backup 5.0.1 or higher.

Requires Solstice Backup Network or Server Edition.

Microsoft Exchange Requirements

  • Target application/database: Microsoft Exchange 4.0 and 5.0

The Solstice Backup Module for Microsoft Exchange Server delivers on-line data protection for Exchange Server applications. Solstice Backup and the Solstice Backup Module for Microsoft Exchange Server combine to provide secure, automated, on-line backup and restore for your critical messaging system -- while it is open and in use. The Solstice Backup Module for Microsoft Exchange Server is an implementation of the Microsoft Exchange Backup/Restore API, providing a high-performance connection between Solstice Backup and Exchange. The Module and the API integrate backup and restore services for Exchange Server Directory and Information Store into Solstice Backup's comprehensive, enterprise-strength storage management solution.

Chapter 4: Data Archiving

Introduction

As the amount of network data grows faster than the storage space available, network managers are looking for ways to maximize their existing storage space without incurring a large amount of cost. To meet this need, a number of software vendors have introduced Hierarchical Storage Management (HSM) products. According to Michael Peterson, President of Peripheral Strategies, Inc. (Santa Barbara, CA), a consulting firm specializing in mass storage issues, "What companies really want (now) is to automate the archive process."

Archiving is the foundation for HSM. Without a sound archiving strategy, HSM is nothing more than a "data mover." Archiving manages the important task of organizing files in a cohesive fashion, such that it facilitates long-term, off-site storage. It is an effective way to free up valuable disk space. Archiving can be viewed as the initial step toward HSM, and may be the final solution for a variety of networked environments.

Data archiving is the process of taking a "snapshot" of a file or a series of related files as it resides on primary media (disk) at a given point in time. The image of the snapshot typically resides on removable media -- usually tapes or optical disk. Once the snapshot is safely stored on removable media, the necessary files can optionally be deleted from the primary storage. Archiving is normally performed on files or data associated with specific projects. Unlike data backup, it is not practical or wise to have a "network-wide, one size fits all" archiving policy. The act of initiating data archiving is controlled by the end user. End users define specific archiving policies that matches their applications needs.

Sun is committed to simplifying the administration of network computing environments, protecting mission-critical data from loss, and managing the data stored on these networks in a consistent and cohesive fashion. Recognizing the immediate need from the user community for a reliable and scalable data archiving solution, Sun offers Solstice Archive software, an add-on application to Solstice Backup server products.

The Need for Data Archival Services

FIGURE  1 Data Archiving

The need for a reliable data archiving and retrieval tool transcends different industry segments and applications. The following scenarios illustrate that practically every network administrator needs to provide a data archiving capability.

Example 1: A CAD engineer working on chip simulation

Bob designs computer chips and uses various ECAD tools to develop and design models. A typical simulation model generates 2000 files and occupies 400 Mbytes of data. Since Bob is constrained by disk space, he cannot keep the output of more than two simulation runs on-line. He wants to archive specific versions of the simulation model before he deletes them from the disk to make room for newer data.

He needs a low-cost, reliable archiving solution that offers the following features:

  • The ability to create a "snapshot" of a chip model at a given point in time.
  • The ability to select a group of files as candidates for archive. Furthermore, to save tape space, the files should be compressed before they're copied to tape.
  • The ability to annotate text that describes the contents of the archive (i.e. "MMU design -- board-level simulation as of July 6, 1994").
  • During retrieval, Bob needs a simple point- and-click approach for retrieving all the files associated with the "July 6, 1994" archive. By default, Bob wants the entire archive data to be retrieved; however, in some cases, he needs the ability to specify a single file contained in the archive set and mark it for retrieval.

Example 2: A financial analyst responsible for creating financial models and monthly financial statements for an organization.

Joan is responsible for managing all the financial data for her organization. The financial data resides on her local hard disk. Joan's machine is backed up on a daily basis. However, as an added measure of security and to conform to auditors' regulations, Joan requires easy-to-use archiving software that enables her to archive relevant financial data on a fairly regular interval. Reliability and data integrity are important needs. Additionally, her key requirements include:

  • A user-initiated archive session that copies all the necessary files to optical media.
  • To make multiple copies of the archived data to different media and track the location of each "archive clone."
  • To verify that the data that has been archived is readable, and that it is the same as what was contained on the disk during the archive process.
  • Archive media should never be recycled.

Example 3: Data analysis in a small pharmaceutical company.

Roger is a technician in a pharmaceutical organization. His job primarily involves data acquisition and analysis. At the end of every work day, Roger wants the ability to "archive" relevant data. His key requirements include:

  • Generate an "archive list" that can be initiated daily by the system administrator at a specific time.
  • His data should be archived to a "specific" set of tapes and should not be mixed with other technicians' data.
  • An extra copy of the archived data must be maintained at an off-site location.
  • If the data is archived and the media is verified successfully, then the data can be removed from the on-line disk.
  • If, during retrieval, the primary media is not available, the software should point the user to all the relevant clones so that the data can be retrieved.

The previous scenarios list just a few examples of the need for data archival. Similar examples abound in every type of industry -- Software Engineering (archiving source code), MCAD (archiving the final design of an airplane wing), Education (archiving research projects and course assignments), Government, Scientific, and Services.

Data Archiving: A First Step to HSM

As previously illustrated, the need for archiving is separate and distinct from data backup and HSM. There are a number of reasons why a sound data archiving strategy is a springboard to implementing an HSM solution. Two important topics are addressed in this section:

Is HSM Really "Infinite" Storage?

There is often a misconception that by implementing HSM, organizations can have infinite storage capacity (to the extent that some HSM software products use the term "infinite storage" as their tag line).

Consider the example of a site that has two levels of hierarchy with a total capacity of 100 Gbytes. What happens if the total network storage exceeds 100 Gbytes? The most obvious answer is: "either spend more money to acquire more storage capacity," or, "archive the data from the optical media to a tape and make room in the optical media." Instead of tackling the problem of archiving after implementing HSM, the site could have provided its users with an archiving tool, so that the amount of data managed by the HSM solution is much more "manageable."

HSM is File-based; Archiving is Project-based

In all of the examples discussed in the earlier section, when users want a retrieval it is often associated with an entire set of files. These files are typically interdependent. If, in the example used above, all the files associated with an archive were actually migrated using an HSM product, then each individual file needs to be de-migrated separately. Contrast this with a typical retrieval operation, where all the files associated with the archive are retrieved in a logical and cohesive manner.

Functional Overview of the Solstice Archive Application

The Solstice Archive application is a client/server application that leverages users' existing investment and the strength of Solstice Backup software and other Solstice Backup related products.

The Solstice Archive server, the basis of the Solstice Archive application, provides file archiving and retrieval services to a range of client machines and is an add-on extension to the existing Legato NetWorker backup server.

The Solstice Archive client, the key functionality included in the Solstice Archiving application, can be any machine on the network that avails itself of the archive services provided by the Solstice Archive server. A Solstice Archive client can exist independently of a Solstice Backup client.

FIGURE  2 Archival Services

Solstice Archive functions can be broadly classified into three categories, as shown below:

  • Data Archival
  • Data Verification
  • Data Retrieval

FIGURE  3 Archival Functions

Data Archival

This is a client-side function that encompasses creating an archive and performing the required post-archiving functions.

Data archiving can be initiated by either an end user or a system administrator. An end user can initiate archives of files and directories that they "own." System administrators have supervisory privileges; therefore, they can archive data residing anywhere on the network. Optionally, system administrators can also provide certain end users with "administrator" privileges for a set of machines.

End-User Archival

A typical archival as initiated by an end user entails the following simple steps:

Using an intuitive graphical user interface, the user marks a set of files or directories that need to be archived. Once the files are marked, the user can provide a "name" to the archive and can also annotate up to 1024 characters to easily identify the archive during a retrieval process. An example of an annotated text would be: "Acme's financial results for June 1994."

The user can also optionally exclude or compress data before it is archived.

The next step would be for the end user to specify that they want an extra copy of the archive to be made on a separate medium. This is in addition to the archive that resides on the original medium.

Users can also specify if they want the archive data to be verified, and also indicate the type of verification (see "Data Verification" on Page 42 for more details).

Post-Archive Activity

The user can either leave files that were archived on-line or optionally remove them after the archive. Solstice Backup storage management will not remove the files unless all the conditions for the archive have been met (for example, it will not remove the files if verification failed).

If a user does not wish to use a Graphical User Interface, then they can use the command line interface to initiate an archive session.

System Administrator's Perspective

Archives performed by system administrators follow the same set of steps with the following exceptions:

Administrators can create archive lists. An archive list can be composed of files and directories from different machines. Archive lists are useful in situations where a specific set of related files needs to be archived on a frequent basis. A simple example of an archive list is:

jupiter/usr/src

pluto:j:marketing

Administrators can schedule an archive session to be initiated at a specific time in the future (end-user archives are performed only on an ad-hoc basis).

Cloning

In addition to the first "clone" that is created as part of an archive, system administrators can create multiple "clones" of an archive post-facto.

Data Verification

Since archived data may eventually be deleted from the system, Solstice Archive provides an extra level of security to make sure that the archived data is verified. Solstice Archive verifies the data in two ways:

Media Verification

This is the process by which Solstice Backup software reads media and ensures that the media is readable and that there are no bad spots on the tape. If an error should occur, it alerts the user.

Data Verification

This is the process by which Solstice Backup software reads the data on the media and compares with what was originally archived. If there are any discrepancies, Solstice Backup alerts the users.

Both Media Verification and Data Verification are performed by the Solstice Backup server.

Data Retrieval

This is a client-side program. When the retrieval program is invoked, the software presents the end user with all the archive lists that were initiated by the end user. This will be presented in a GUI-based layout. By default, the end user can view all the archive lists performed from their machine. End users can also "sort" through their archive lists based on certain selection criteria. For example, the user can specify display of all archives performed between June 1, 1994 through June 30, 1994. It will display the relevant archives along with the "annotated" text for that archive. Furthermore, the user can also sort based on the annotated text (i.e. show all archives that contain the text "Acme's financial").

Once the user decides which version of an archive list to retrieve, they can retrieve an entire archive set or specific files within the archive. The retrieval component of the software provides options for relocating, renaming, or overwriting retrieved files.

If the primary media which contains the archive is not available, Solstice Backup software will list all of its appropriate media clones. If any of the clones are available, Solstice Backup will automatically retrieve the data from a clone.

End users can retrieve archives that they "own." System administrators have supervisor privileges; hence, they can retrieve archives for different client machines.

Solstice Archive Application -- Implementation Overview

This section provides an overview of Sun's implementation of archival services.

Server Extension

The server component of Solstice Archive is an add-on extension to the Solstice Backup server. Solstice Backup version 5.0.1 is "archive-ready." It has the necessary hooks built-in that understand the notion of a backup save set and an archive save set. By installing an appropriate enabler code, a Solstice Backup server can also provide Archive services.

The server manages the backup save set and archive save set as separate entities. From the Solstice Backup perspective, a backup save set and an archive save set differ in the following ways:

  • The physical media that contains backup save sets is separate and distinct from the media that contains archive save sets. In other words, archive save sets can only be directed to "archive pools." Backup data and archive data cannot reside on the same tape.
  • The life of an archive save set is "forever." This implies that unlike a backup save set which has "finite" retention times, the default retention time for an archive save set is "forever."

Media Pools and Cloning -- A Powerful Combination

A simple example will illustrate how the media pools and the cloning feature can be used in tandem to manage archive data efficiently.

In this example, a Solstice Backup server has two autoloaders. The first autoloader contains tapes that belong to the pool "archive." The second autoloader contains media that belongs to the pool "archive_clone." When a user initiates an archive, the data is directed to a media belonging to the pool "archive." After the archive is complete, Solstice Backup automatically clones the data contained in the media belonging to the pool "archive" to a media belonging to the pool "archive_clone." Over time, there will be two copies of archives on two separate media. If the system administrator so desires, they can ship the tapes belonging to the pool "archive_clone" off-site for extra protection.

Availability

The Solstice Archive module for SPARC and x86 systems requires Solstice Backup 4.1.2, 4.2, 4.2.6, or 5.0.1 (Network Edition plus Solstice Backup Turbo or Server Edition). All components are currently available.

Conclusion

Sun believes that it is very important for network administrators to adopt a sound data-archiving strategy as a first step toward implementing an HSM solution. Sun's archiving strategy is to offer add-on extension to its existing backup and recovery product. This strategy allows users to preserve their existing investment in storage, and also allows them to capitalize on Solstice Backup strengths for enterprise data management -- namely performance, heterogeneity, reliability, ease-of-use, and ease of data retrieval.

The Solstice Archive application is the tool that will enable users to evolve towards an HSM solution. Solstice HSM is available for those users ready to add HSM to their growing number of storage management tools.

Chapter 5: Solstice HSM

Introduction and Overview

Data storage in corporate LANs and WANs is growing at a explosive and sometimes alarming pace. It is estimated that a typical network has experienced a three-fold increase in data storage between 1991 and 1995. The dramatic decrease in prices of hard disk storage and the unprecedented growth of networks and new applications such as multimedia has fueled this explosion even more.

This explosion in data storage has posed challenging problems for network and system administrators. Initially, system administrators threw more hardware at the problem to satisfy the requirements from the users for more disk space. In the last few years, IS personnel -- especially those who have been exposed to the data center/mainframe storage management concepts -- have started to explore and deploy Hierarchical Storage Management (HSM) solutions for the client/server distributed computing environment.

Today, it is estimated that there are only a few hundred sites worldwide that are currently deploying HSM to manage storage for mission-critical, client/server environments. A vast majority of sites that have deployed HSM for their client/server environment are often disappointed with the results.

The reasons for this low level of acceptance are many, but two important factors emerge:

  • HSM was thought of as a panacea to all storage management problems and vendors promised dramatic reduction in cost of ownership; "just install this software and all your storage management problems go away."After deployment, IS personnel found that they traded in one set of problems for another, and mission-critical data was not being protected or managed effectively.

  • Another important factor was that after deployment, IS personnel realized that the dynamics of the mainframe/data center environment are completely different from the distributed client/server environment. Issues such as heterogeneous platforms and operating systems, network bandwidth, and tape speeds were not considered before deployment. As a result, users found themselves in situations where the HSM solution that was touted as a corporate standard managed the storage on only one of the 15 different platforms that existed in the distributed environment.

This white paper is a primer on Hierarchical Storage Management (HSM) for the distributed client/server environment. It provides information on basic HSM definitions, benefits of HSM, the role HSM plays in overall storage management, and factors to consider before deploying HSM. The final section of this paper contains information on Sun's HSM application, including implementation and a statement of direction on future plans.

HSM -- A Few Definitions and Concepts

The operative words in the phrase Hierarchical Storage Management (HSM) are Hierarchy and Storage. HSM is a policy-driven data management strategy whereby data is moved from one storage medium to another based on a set of policies. The hierarchy of storage media is often a function of cost of media (in $/MB) and access speed of data from the specific media. Typically, disk is the first stage of hierarchy and tape is the last stage.

  • Migration Client: A migration client is any system on the network which contains data that needs to be migrated -- either now or in the future. A migration client consists of multiple file systems or volumes, and either one or all volumes can be under migration control.

  • Migration Server: A migration server is a system on the network that provides migration services to clients on the network.

  • Migration Store: The migration store resides on the migration server. It contains data that originally resided on the migration clients before being automatically moved over to the server. The migration store can be any type of storage media        -- disks, tapes, optical, etc. Typically, the migration store consists of a hierarchy of storage media and the moves across this hierarchy, which are based on certain staging policies.

  • File Migration: This in an operation where files are migrated from one physical location to another based on migration policies set by administrators. Migration policies define the criteria for migrating files (e.g. when should files be migrated from a system, and what specific selection criteria should be used to select files for migration).

    The answer to when files should be migrated from a system are controlled by water marks for each file system that is under migration control. A high water mark defines the critical phase when a file system or disk reaches is critical level of capacity. The selection criteria is often based on access time (migrate a file if it has not been accessed for over 90 days), size (migrate files that are over 30 Mbytes) and file owners (migrate Bobs files). Migration occurs until the file system reaches the low water mark

    File Migration can be automatic or on-demand. Automatic file migration is initiated when a threshold as defined in the migration policy is reached. This operation is completely transparent to the user and the administrator. On-demand migration is initiated by the system administrator who needs to migrate certain files and cannot afford to wait until the high water mark is reached.

  • File Recall or De-migration: File migration, whether automatic or on-demand, is completely transparent to the user. Even after the file is migrated, it appears to be local to the user. When the user needs to access the file for use, it is recalled or de-migrated automatically from the migration store. In some cases, users might notice a pronounced delay when trying to access the file since the migrated store can be a tape which has a latent delay in access rate when compared to disk.

    Data Staging: Staging is defined as the process where the migrated data is moved from one level of storage hierarchy to the next within the migrated store. Staging can be one or many levels. As before, the operation of staging is transparent to the user and is governed by staging policies. Staging policies dictate when the data is staged from one storage hierarchy to the next. Typically it is also based on time, i.e. "within the migrated store, move data from optical to tape if it has resided on optical for more than 30 days."

    Hierarchical Storage Management is a judicious combination of migration and staging.

Why Deploy HSM?

Given the number of vendors selling HSM solutions, there are a lot of myths and misconceptions as to why an organization should implement HSM for their distributed environment.

HSM should not be deployed for the following reasons:

  • Substitute for Backup and Archiving: This is the most common trap for IS personnel. HSM is not a replacement for backup and archiving. It is complimentary to backup and archiving and must be viewed as such. While the distinction between backup and HSM is clear, the difference between archive and HSM is often murky.

    Data archiving is the process of taking a "snapshot" of a file or a series of related files as it resides on primary media (disk) at a given point in time. The image of the snapshot typically resides on removable media -- usually tapes or optical disk.

    Archiving provides an extra level of protection for a company's mission-critical data. An important side benefit of archiving is that once the snapshot is "safely" stored on removable media, the necessary files can optionally be deleted from the primary storage. This approach can be used to conserve hard disk capacity. Archiving is normally performed on related files or data associated with specific projects.

  • Reduce Hardware Storage Costs: Some IS personnel have deployed HSM with the notion that they would never have to purchase incremental hard disk storage capacity. Justifying a HSM solution based on a media cost optimization scheme is a moving target -- given the rate at which street prices of storage hardware have been decreasing.

HSM should be deployed with the following in mind:

  • A Complementary Solution to Backup and Archive: By deploying HSM, organizations not only protect the network against accidental data loss, but also allow the system manager to manage the network storage resources more effectively.

  • Housecleaning of Old Files and Data: HSM should be viewed as a storage management policy that enable organizations to effectively manage data on the network by moving old unused data to the migrated store. Any cost savings from storage hardware should be viewed as a secondary benefit. The primary benefit is migrating old files so that newer files can be better managed without compromising the availability of the migrated file.

When Should HSM be Deployed?

Since distributed computing is extremely heterogeneous in nature, organizations should ensure that certain basic storage management techniques are in place before HSM is deployed enterprise-wide. These include:

  • Sound Backup and Archiving Strategy: As mentioned earlier, HSM is not a replacement for backup or archiving. Before HSM is deployed, it is recommended that organizations have a sound backup and archive strategy in place for data protection across the heterogeneous network.

  • Size of Networks and Age of Data: Typically, large networks with an inordinate amount of old data are ideal candidates for deploying HSM. While it is tough to quantify old data, it is recommended that organizations deploy HSM if data older than 1 year takes up a sizable chunk of the network storage resources. There are several tools available on the market that analyze the age of data.

  • Commitment from End Users: It is imperative that IS personnel gain full support from their end-user community before deploying HSM across the organization. The good and bad news with the file migration component of HSM is that it is transparent to the end user. End users are often irritated to find that their data has been migrated to a different place without their knowledge. This is exacerbated when they try to access a migrated file and it takes longer than usual to access it (due to the built in latency in de-migrating the file).

Factors to Consider When Recommending an HSM Solution

To ensure a successful integrated storage management strategy, organizations should take into consideration various issues before selecting a HSM solution. Key factors include:

Integrated Solution: Organizations should, when possible, implement backup, archive, and HSM solutions from the same software vendor. Buying the integrated solution from the same vendor has many advantages. A few such advantages are:

  • A consistent method to manage the corporate data. After all, it's the same data that needs to be managed differently.
  • Effective sharing of storage resources (the same autoloader can be used for backup as well as migrated store).
  • A common user interface.
  • A single point of contact for technical support and other support services.

One key advantage of deploying the solution from one vendor is a requirement for peaceful coexistence between backup/archive and HSM. If the solution is not integrated, users might find that backing up a file causes recall or de-migration and the migrated store cannot be backed up using the backup product.

  • An Enterprise Problem Requires an Enterprise Solution: Heterogeneity and distributed client/server computing go hand-in-hand. It is critical that the HSM solution being deployed supports (or the vendor plans to support) all the key platforms in the network.

  • Stick with the Standards: One of the biggest stumbling blocks for HSM deployment is that very few operating system vendors provide inherent support for file migration as part of the standard operating system. This has necessitated third-party vendors to force users to modify file systems or kernels to facilitate migration. Changing the underlying file system to support a third-party HSM solution is not a recommended path since changing the file system means you are no longer running a OS that is supported by the OS vendor. Vendors such as SGI and Novell (4.1) have file systems that are HSM-ready. Standards such as DMIG (for UNIX) and RTDM (for NetWare 4.1) are facilitating a common interface that would enable other OS vendors to make their file system HSM-ready. It is recommended that a vendor's deployment is not proprietary and they have a strong presence in standards efforts.

  • Independence from Storage Hardware: The best software for HSM should work with any storage media including tape, optical, or magnetic disk. It should not force the user to purchase a specific hardware (e.g. works only with optical jukebox from vendor xyz).

Product Overview

The Solstice HSM server provides file migration and recall services to a range of client machines. HSM is packaged as an optional extension to existing Solstice Backup servers and uses the same license mechanism as Solstice Backup.

A Solstice Backup HSM client can be any machine on a network that uses file migration and recall services provided by an HSM server. Clients may be enabled for backups, archives, HSM, or all three. Note: Solstice HSM is supported for Solaris clients only.

Based on input from customers, several goals were selected in designing the HSM support. These goals were:

  • Automatically migrate inactive data to less expensive storage across multiple platforms based on administrator-selected criteria: These criteria include high and low watermarks, minimum file size, days since last reference (read or write), file owner or group, and file inclusion or exclusion. Clients and save sets meeting these criteria are available for pre-migration. Pre-migration copies the file to the storage location leaving the original on the clients local disk. When Solstice Backup backs up a group containing migration save sets, pre-migration occurs. This happens automatically and is not controlled by high and low watermarks. During normal operations a migration daemon monitors the file system, and when high watermark is reached or the local disk is full, migration occurs; the pre-migrated file is quickly deleted from the disk, leaving a stub which contains information about the migrated file.

  • Support a hierarchy of hard disks, optical jukeboxes, and tape libraries: HSM supports all devices supported by Solstice Backup, including tape and optical libraries. The libraries can be shared with backup and archive data, or they can be separate. Solstice HSM has the ability to backup directly to magnetic disk devices, enabling multi-level staging from disk to disk and then from disk to optical or disk to tape. Instead of migrating data directly to removable media, you can move data (using save set staging) directly to other primary media (disk). Disk to disk save set staging with HSM results in higher performance data retrieval. You can still create a set of policies to migrate the data to removable media, effectively creating multiple levels of HSM.

  • Provide transparent user recall of migrated data: After a file migrates and is replaced with a stub, the user can perform the same actions on it as on any other file in the file system. The file can be moved, renamed, and any other action that does not require read or write access. When a user accesses a migrated file to read, write, or change attributes, Solstice Backup recalls the file to the location of the stub. After the recall starts, the file begins to open. The recall operation takes place concurrently with the opening of the file. Therefore, the user notices a delay in reading and writing until the file is completely recalled to its original position. When a file is removed from the file system name space, it is also removed from the migration store.

  • Provide integration with backup operations: Solstice Backup recognizes migrated stub files, and will only backup the stub; it will not cause a file recall. Restoring a migrated file will restore the stub, but it won't recall the migrated file. Additionally, Solstice Backup can perform super-fulls for HSM clients. A super-full is a clone (or copy) of the most recent full backup of a save set and a clone of all migration save sets. A super-full contains a complete backup of the data on the client and the data in the migrated store. This is very important if you ever need to recover from a disaster.

HSM support is available on SunOS 4.1.4 and Solaris 2.3, 2.4, 2.5, and 2.6 client platforms. Additionally, several of our OEM partners will be providing HSM support for their client systems.

Sun also provides Solstice Storage Management Suite for Solaris, an integrated suite of storage management applications all managed from a consistent administrative interface. Solstice Storage Management Suite integrates and automates backup, archival, and file-migration operations.

Product Implementation

Enabling HSM support is as easy as the administrator establishing a few basic policy options for the client system (see Figure 1). Once enabled, the local hard disk is monitored according to these policies, with files automatically migrated to the server based on usage policies and recalled transparently when referenced. The administrator can easily monitor HSM activity for all clients (see Figure 2) and change these policies as required to optimize system and storage resources.

FIGURE  1 HSM Migration Administration Window

Using the Motif-based Administration GUI, the Solstice Backup administrator configures the policies for client HSM support. These include:

  • The minimum and maximum amount of guaranteed free space to keep available on the clients local file system (for example, if the file system exceeds 90% full, migrate files until it reaches 70% full).
  • A list of files that must never be migrated (the base operating system files are never migrated).
  • A list of file systems to place under HSM control (for example, ALL or /opt, /home).
  • The minimum size of a file before it can be considered for migration (for example, don't migrate files less than 2 Mbytes in size).
  • A list of file owners and group owners whose files must be included or excluded from migration (for example, exclude all files owned by operator).

FIGURE  2 HSM Migration Control Window

Solstice HSM manages your clients data, providing a virtually unlimited disk capacity. During normal backup operations it transparently pre-migrates eligible files based on administrator-defined storage policies. When space runs low, it will convert these files to a symbolic link, allowing files to be accessed in the same manner as any other standard file; it will transparently recall the file in the background.

Solstice Backup tracks the number and size of pre-migrated, migrated, and recalled files for each client's file system. Using this information, the administrator can easily adjust client policies to optimize data storage resources and overall system performance.

Statement of Direction

As part of our on-going commitment to providing high-quality products to our customers, Sun will continue to enhance the functionality of Solstice HSM software.

These statements represent current development plans of Sun. Development plans are subject to change or cancellation without notice. Reliance on this Statement of Direction is solely at the risk of the relying party and does not create any liability or constitute an obligation for Sun in any manner implied or assumed.


1 -- Solstice Backup is based on Legato NetWorker for Solaris software from Legato Systems of Palo Alto, CA.
Evaluate
»   Trial Downloads
»   Promotions
»   Tours
»   Software Collateral
»   Case Studies
»   White Papers
»   News & Events
 
 
 
Get
»   Get Software
»   Training
»   Buy Software Online
 
 
 
Use
»   Documentation
»   FAQs
»   Communities & Resources
»   Developer Resources
»   Newsletters
 
 
 
Maintain
»   SunSolve Patches
 
Give your business a boost
Contact About Sun News & Events Employment Site Map Privacy Terms of Use Trademarks Copyright 1994-2008 Sun Microsystems, Inc.