by Dean Halbeisen
This article is Part 3 of a three-part series that describes how Oracle Optimized Solution for Secure Backup and Recovery delivers end-to-end data protection with high availability and security for Oracle's engineered systems and other Oracle Optimized Solutions.
|
This article describes best practices for backing up Oracle engineered systems and provides pointers to more-detailed information.
The articles in this series are located here:
| |
|
Oracle Optimized Solutions provide tested and proven best practices for how to run software products on Oracle systems. Learn more.
|
|
Table of Contents
Best Practices for Oracle Engineered Systems Backups
The following sections detail best practices for backing up Oracle engineered systems and provide pointers to more detailed information.
Oracle SuperCluster and Oracle MiniCluster Backup Best Practices
Oracle Optimized Solution for Secure Backup and Recovery can be used as a complete solution for performing backups of Oracle SuperCluster and Oracle MiniCluster systems. Leveraging built-in integration of Oracle hardware and software, it provides pretested, recommended solutions for backup and recovery and has the following characteristics:
- Comprehensive. This solution provides complete end-to-end data protection for Oracle SuperCluster, and it supports a wide range of backup options including disk, tape, and cloud-based backups.
- Secure. Oracle Optimized Solutions leverage Oracle's SPARC-based systems, which offer high-performance security using cryptographic instruction accelerators that are directly integrated into the processor cores.
- High performance. Native high-performance networking built into Oracle SuperCluster engineered systems provides faster backup and recovery of applications and databases. Inherent scalability and no single point of failure help ensure service-level agreements (SLAs) are met as data capacity grows.
- Lower cost. This solution supports disk-only backup and recovery options that do not rely on tape backups. These configurations eliminate the need to license additional tape management software, thereby reducing costs.
- Simplified. This solution uses tested configurations that have been optimized for Oracle hardware and software, and Oracle Enterprise Manager provides end-to-end management and monitoring. Consolidation of disparate legacy backup solutions onto a single multipurpose backup and recovery platform can eliminate the need for specialized backup and recovery hardware.
For more information on Oracle SuperCluster backup and recovery, please see the Oracle Technical Network article "Using Oracle Optimized Solution for Secure Backup and Recovery with Oracle SuperCluster."
For more information on Oracle MiniCluster backup and recovery, please see the Oracle Technical Network article "Using Oracle Optimized Solution for Secure Backup and Recovery with Oracle MiniCluster."
Oracle Exadata Backup Best Practices
The Oracle Maximum Availability Architecture (MAA) group develops Oracle best-practices blueprints for Oracle Exadata backup and recovery. The group has published a white paper, "Backup and Recovery Performance and Best Practices for Exadata Cell and Oracle Exadata Database Machine," which details best practices for backup and recovery of Oracle Exadata systems. The paper contains performance results for various Oracle Exadata configurations using both disk and tape backups. With detailed steps necessary for configuring the database, Oracle RMAN commands, and network settings for optimal backups, the white paper also includes recommended strategies for restore and recovery operations. In addition, readers can learn more about offloading backups using the Data Guard feature of Oracle Active Data Guard and about gaining additional benefits through the use of Oracle Active Data Guard.
Oracle Exadata systems can be backed up to tape or disk. While both media types provide extremely fast backup and restore rates, there are advantages to each. Tape-only solutions isolate faults from the Oracle Exadata Storage Servers and maximize Oracle Exadata capacity and bandwidth. Disk-only backups provide faster recovery times for data and logical corruptions and some tablespace point-in-time (TSPITR) scenarios. Disk backups also provide the ability to use backups directly with no restore by switching to a copy of the database, tablespace, or data file.
Oracle Database Appliance Backup Best Practices
As with the other Oracle engineered systems, both disk and tape backups provide extremely fast backup and restore rates for Oracle Database Appliance, but there are advantages to each. Tape-only solutions isolate faults from Oracle Database Appliance and maximize appliance capacity and bandwidth. Disk-only backups provide faster recovery times for data and logical corruptions and some tablespace point-in-time (TSPITR) scenarios. Backups to disk also provide the ability to use backups directly with no restore by switching to a copy of the database, tablespace, or data file.
Configuring Oracle Database Appliance for Local and External Backups
Choosing external versus local backup in the Oracle Database Appliance Manager during deployment will affect the size of the disk groups +DATA and +RECO, which will determine the FRA size. When configuring the appliance, selecting the external backup type option in the Oracle Database Appliance Manager Configurator utility will assign 80 percent of the disk to the DATA area and 20 percent of the disk to the RECO area. Selecting the local backup type option in the utility assigns 40 percent of the disk to the DATA area and 60 percent of the disk to the RECO area.
For More Information
For a more detailed discussion of the best practices for backing up Oracle Database running on Oracle Database Appliance, refer to section "Best Practices for Database Backups (Oracle Database 11_g_ Release 2 or Higher)" in Part 2 of this series.
The Oracle white paper "Backup and Recovery Best Practices for the Oracle Database Appliance" contains details on recommended architectures, suggested configuration parameters, best practices for backup and recovery of Oracle Database Appliances, and a detailed section on backing up appliances to network-attached storage (NAS) devices. Also included are performance results for Oracle Database Appliance configurations using local or external disks as backup targets, and the detailed steps necessary for configuring the various appliance components in order to obtain optimal backup results. For more information, please see the white paper, or refer to My Oracle Support Note 1558851.1. For detailed information on tape backups, refer to the Oracle white paper "Protecting Oracle Database Appliance — Tape Backup with Oracle Secure Backup."
Best Practices for Oracle ZFS Storage Appliance Systems
Oracle recommends certain procedures for optimal configuration and performance of Oracle ZFS Storage Appliance systems with Oracle SuperCluster or Oracle Exadata. Some of these procedures are outlined here.
Use the Oracle Engineered Systems Backup Utility for Oracle ZFS Storage Appliance Systems
Oracle recommends the use of the Oracle Engineered Systems Backup Utility for Oracle ZFS Storage Appliance systems. This tool, which can be used to configure the Oracle ZFS Storage Appliance system that is internal to the Oracle SuperCluster and any additional external Oracle ZFS Storage Appliance systems used for backup, verifies the system configuration and automates the necessary configuration steps, including the following:
- Configuring Oracle ZFS Storage Appliance systems. This includes creating a project and shares for Oracle RMAN.
- Configuring the Oracle SuperCluster database nodes. This includes enabling Direct NFS Client, creating mount points, and adding backup services.
- Creating the final Oracle RMAN scripts. (Note: The Oracle RMAN run block scripts are intended for samples only, and should be reviewed and modified as necessary to meet site-specific requirements.)
Although these configuration steps can be performed manually, use of the Oracle engineered systems' backup utility is recommended to reduce the risk of accidental misconfiguration. The backup utility for Oracle ZFS Storage Appliance systems can be downloaded from the Oracle ZFS Storage Appliance Plug-in Downloads web page.
Network Configuration Best Practices
Network configurations for Oracle ZFS Storage Appliance systems vary according to the environment where the appliances are deployed. If an Oracle ZFS Storage Appliance system is to be used to back up one or two Oracle engineered systems and it is located within 100 meters of the Oracle engineered systems, then InfiniBand networking can be used to connect all the systems. With this recommended configuration, each Oracle engineered system resides on its own InfiniBand fabric and a different InfiniBand subnet.
When used with Oracle SuperCluster, an Oracle ZFS Storage Appliance system can be connected directly to the Oracle SuperCluster InfiniBand leaf switches if the Oracle ZFS Storage Appliance system is the only other appliance or device (other than Exadata Storage Expansion Rack) that will be connected to the infrastructure. An Oracle ZFS Storage Appliance system can be connected to external InfiniBand leaf switches if more appliances or devices will be connected. If an Oracle ZFS Storage Appliance system is to be used to back up more than two Oracle engineered systems, or an Oracle ZFS Storage Appliance system cannot be located within 100 meters of the Oracle engineered systems, then 10 GbE should be used to connect the Oracle engineered systems to the Oracle ZFS Storage Appliance system. The following are additional recommendations:
- Configuring the network for InfiniBand connectivity: To provide the highest availability and performance, Oracle recommends that each Oracle ZFS Storage Appliance controller be configured with two dual-port QDR InfiniBand HCA cards, providing a total of four ports per Oracle ZFS Storage Appliance controller. The controllers should also be connected to a pair of redundant 10 GbE switches to provide continuous connectivity in the event of an outage. This configuration requires the use of an active/standby IPMP group. For configuration details, see the Oracle white paper "Configuring a Sun ZFS Backup Appliance with Oracle SPARC SuperCluster."
- Configuring the network for 10 GbE connectivity: To provide the highest availability and performance while not using the InfiniBand network, Oracle recommends that each Oracle ZFS Storage Appliance controller be configured with two dual-port 10 GbE optical cards, providing a total of four ports per Oracle ZFS Storage Appliance controller. This configuration requires the use of an active/standby IPMP group. Additionally, for performance reasons, the 10 GbE network should be configured with a large MTU size, also known as jumbo frames. For configuration details, see the Oracle white paper "Configuring a Sun ZFS Backup Appliance with Oracle SPARC SuperCluster."
- Configuring network cluster resources: Once the network resources have been configured for either 10 GbE or InfiniBand, the interfaces need to be placed under the control of the Oracle ZFS Storage Appliance cluster. Clustering the network resources provides continuous access to the data on the Oracle ZFS Storage Appliance system in the event of a controller head failure in the appliance.
- Configuring networking for the Oracle ZFS Storage Appliance system: The InfiniBand ports on the Oracle ZFS Storage Appliance system must be configured for IP multipathing (IPMP). Four IP addresses will be required for each Oracle ZFS Storage Appliance head—for a total of eight addresses—because the interfaces will be running in an active-active configuration.
For more detail, and a list of principles to use for database restore operations, see the Oracle white paper "Backup and Recovery Performance and Best Practices Using the Sun ZFS Storage Appliance with the Oracle Exadata Database Machine." Please refer to My Oracle Support Note 1354980.1 to get details on the recommended software stack for Oracle ZFS Storage Appliance systems when used with Oracle Exadata. Refer to the Oracle white paper "Configuring a Sun ZFS Backup Appliance with Oracle SPARC SuperCluster" for more details on how to configure Oracle ZFS Storage Appliance systems for use with Oracle SuperCluster to provide maximum throughput and availability.
Storage Configuration Best Practices
Configuring Oracle ZFS Storage Appliance storage pools assigns physical disk drive resources to logical storage pools for backup data storage. When backing up and recovering Oracle Database to an Oracle ZFS Storage Appliance system, Oracle recommends that Oracle ZFS Storage Appliance disk resources be configured in two equally sized pools. Configure the two pools by assigning half of the physical drives in each drive tray to each storage pool to maximize system throughput.
Once the pools have been created, they should be placed under the control of the Oracle ZFS Storage Appliance cluster to protect against the failure of a network component supporting the appliance or a component within the appliance's controller. Doing so ensures that an Oracle RMAN database backup or restore operation can continue to run. For more details, see the Oracle white paper "Backup and Recovery Performance and Best Practices Using the Sun ZFS Storage Appliance with the Oracle Exadata Database Machine" or the Oracle Technical Network article "Using Oracle Optimized Solution for Secure Backup and Recovery with Oracle SuperCluster."
Configuring Oracle RMAN Projects and Shares on Oracle ZFS Storage Appliance Systems
Share configuration is the process of setting up and running NFS mount points for client access. A project is an entity that provides a higher-level management interface point for a collection of shares. Two projects should be created for an Oracle Database 11_g_ Release 2 configuration on Oracle SuperCluster—one project per pool. To optimize share management, update the default mount point for shares contained in the project to reference the database name. To optimize a system for performance, create four shares for each project in each pool, for a total of eight shares (four on each head).
Oracle provides the Oracle Engineered Systems Backup Utility—downloadable from Oracle Technology Network at the Oracle ZFS Storage Appliance Plug-In Downloads page—that automates the process of creating Oracle ZFS Storage Appliance projects and shares and creates the corresponding entries. The utility also creates Oracle RMAN command files that can be used for backing up the database. Running the utility ensures that all of the following best practices of using an Oracle ZFS Storage Appliance system as an Oracle Exadata backup destination are followed:
- Create two projects, one per pool.
- Configure the projects to use a write bias of "throughput."
- Configure the project's NFS exceptions to allow access to the shares by the Oracle engineered system.
Database Backup and Restore Best Practices with Oracle ZFS Storage Appliance Systems and Oracle Engineered Systems
The following key practices should be implemented in order to obtain maximum availability and performance from Oracle ZFS Storage Appliance systems. Note, however, that for any backup operation or changes to backup practices, IT staff should perform testing to ensure that the backups complete within the available backup window and that the database restore and recovery operations can meet recovery time objective requirements.
Configure Network Connectivity
Do the following to configure network connectivity:
-
Enable Direct NFS Client. An Oracle Database instance running on an Oracle engineered system is backed up to an Oracle ZFS Storage Appliance system using the Direct NFS Client feature of Oracle Database 11g Release 2. Note, however, that by default, the Direct NFS Client feature is not enabled, and enabling it requires a database shutdown in order to relink the Oracle Database kernel. By enabling Direct NFS Client, the operating system kernel NFS processes are bypassed and the Direct NFS Client opens up 1 MB network buffers solely for use by Oracle RMAN, bypassing the system-configured TCP network buffers and speeding up the database backup and restore. For details, see the Oracle white paper "Backup and Recovery Performance and Best Practices Using the Sun ZFS Storage Appliance with the Oracle Exadata Database Machine" or the Oracle white paper "Oracle Optimized Solution for Secure Backup and Recovery: Oracle SuperCluster Backup and Recovery."
-
Configure oranfstab. It is not essential to create an oranfstab file for Oracle Database to utilize Direct NFS Client. However, when using any of the following configurations, an oranfstab file must be created to fully utilize the network resources between the database nodes in Oracle Exadata or Oracle SuperCluster and the Oracle ZFS Storage Appliance system:
- An Oracle engineered system is to be connected to the Oracle ZFS Storage Appliance system using InfiniBand
- An Oracle engineered system is running Oracle Solaris
- An active/active IPMP group is created on an Oracle ZFS Storage Appliance system
Refer to My Oracle Support Note 1517107.1, "RMAN Backup from SPARC SuperCluster to Oracle ZFS Storage Appliance," for sample /etc/oranfstab files.
Follow Recommended Best Practices for Database Backup and Restore
Oracle recommends using a combination of level 0 and level 1 backup sets when backing up Oracle Database to an Oracle ZFS Storage Appliance system. The frequency of backups is dictated by recovery time and recovery point objective requirements. Note that when using an Oracle ZFS Storage Appliance system, Oracle does not recommend an incrementally updated backup strategy using a combination of data file copies and incremental backup sets that are subsequently merged into the data file copies. When an incrementally updated backup strategy is selected, the backup solution becomes I/O-bound during the merge process, and alleviating this bottleneck requires a significant number of available disk spindles in order to achieve the required IOPS rate. For most IT departments, the following recommendations should be sufficient:
- The Oracle Database FRA should remain on the Oracle Exadata Storage Servers that are part of the Oracle Exadata or Oracle SuperCluster and should not be put on the Oracle ZFS Storage Appliance system.
- The weekly full level 0 backup should be written to the Oracle ZFS Storage Appliance system.
- A daily incremental level 1 backup should be written to the Oracle ZFS Storage Appliance system. A maximum of 16 Oracle RMAN channels should be allocated and configured for both weekly level 0 and daily level 1 backups to write to one of the 16 Oracle ZFS Storage Appliance shares created using the Oracle Engineered Systems Backup Utility mentioned earlier (or created manually).
- Two separate backup jobs should be submitted if Oracle RMAN compression is used to preserve space on the Oracle ZFS Storage Appliance system but the database consists of a mixture of uncompressed data and compressed data or data that cannot be compressed further. One of the backup jobs will create an uncompressed backup, and the second will create a compressed backup. Otherwise, compressing data that does not typically yield good compression lengthens the backup window significantly, consumes unnecessary CPU resources, and results in very little space savings.
- Oracle RMAN offers an option to amortize the backup over a given time period and minimize the performance impact to critical applications. For example, if there is an eight-hour backup window available for the weekly level 0 backup of a 20 TB database, but critical database functions are still running during that backup window, the Oracle RMAN
duration minimize load option can spread the work over the entire backup window.
- The use of Oracle RMAN options such as
sectionsize is recommended. These options are included in the backup scripts generated by the Oracle Exadata Backup Configuration Utility for Oracle ZFS Storage Appliance. The sectionsize parameter breaks up BIGFILE tablespaces into more manageable amounts. Oracle recommends that the largest tablespaces be broken into equal-size sections for each Oracle RMAN channel. For instance, if there are two tablespaces of 10 TB and 14 TB, and there are 16 Oracle RMAN channels, then the sectionsize parameter should be set to approximately 320 GB.
- The Oracle Exadata Backup Configuration Utility for Oracle ZFS Storage Appliance does not enable Oracle RMAN or Oracle ZFS Storage Appliance compression, because this is something that is dependent on the data to be backed up and the database licensing options purchased. Enabling Oracle RMAN compression increases CPU utilization.
Tune the Oracle Database Instance for Oracle RMAN
Optimizing high-bandwidth backup and restore operations using Oracle RMAN and an Oracle ZFS Storage Appliance system requires adjusting the instance parameters that control I/O buffering. For information about how to tune these parameters, see My Oracle Support Note 1072545.1 "RMAN Performance Tuning Using Buffer Memory Parameters" at support.oracle.com.
Configure Oracle RMAN
Performance benefits can be realized by configuring 16 or 32 Oracle RMAN channels to be evenly distributed over the Oracle Database instances and nodes in the Oracle RAC cluster. The channels also should be evenly distributed over the shares exported from the Oracle ZFS Storage Appliance system.
Best Practices Using Oracle ZFS Storage Appliance Systems
Traditional backups are performed with a base-level full backup and then backups are done at subsequent intervals to capture only the files that have changed. This can be an inefficient method, because restore operations must start with the full backup and apply each incremental backup in the proper order. Also, any and all incremental backups must be applied or the restore will not be complete.
When using external Oracle ZFS Storage Appliance systems for backups, it is possible to follow a particular strategy shown in Figure 1. Backup operations should start with a full level 0 backup on the first day. On the second day, IT staff can perform an incremental (level 1) backup to record just the blocks that have changed from the previous day, or they have the option to apply the level 1 incremental to the full level 0 backup, as shown in Figure 2 (using Day 3 as an example). The incremental applied backup merges any changes since the full backup and creates a single restore point. This is a much more efficient method than the traditional combination of full and incremental backups typically used to obtain an up-to-date restore.
Figure 1. Oracle's suggested backup strategy for databases.
Use ZFS Snapshots
The previous procedure can be taken a step further by utilizing the snapshot capabilities of Oracle ZFS Storage Appliance systems, as illustrated in Figure 2. It starts again with a full level 0 backup on Day 1 and an incremental backup on Day 2. On Day 3, the incremental backup is applied to the level 0 backup from the first day, and a snapshot is taken of the applied incremental, resulting in an image that looks like a version of a full backup. This consumes a lot less space because it is recording only changes.
Figure 2. Oracle's suggested backup strategy with Oracle ZFS Storage Appliance systems.
Using Non-Oracle Exadata Storage for Oracle SuperCluster and Oracle Exadata Database Machine Backups
While they are not recommended, non-Oracle Exadata storage solutions might be necessary in some cases. Differences in network connectivity, storage configuration, and protocols used by non-Oracle Exadata storage will affect backup performance and contribute complexity. The Oracle Maximum Availability Architecture white paper "Backup and Recovery Performance and Best Practices for Exadata Cell and the Oracle Exadata Database Machine" contains suggestions on how to avoid the pitfalls of non-Oracle Exadata storage in these environments.
For configuration details and up-to-date changes, please see My Oracle Support Note 1558851.1.
For more information on Oracle Maximum Availability Architecture, please see "Best Practices Blueprint for High Availability."
Providing Backup and Recovery Savings and High Availability with Remote Disaster Recovery Sites
Figure 3 shows the ultimate solution for providing the ability to restore IT operations while spending less for backup and recovery. Traditional solutions require expensive backup and recovery equipment to restore the production system as quickly as possible. An alternative solution is to have a second remote site with replicated applications and data. Using primary and secondary remote sites eliminates the need for high-performance systems to recover production operations. Instead of deploying Oracle SuperCluster and a high-performance disk array to back it up, a secondary Oracle SuperCluster with Oracle's lower cost Oracle ZFS Storage Appliance system or tape can be deployed. In the event of a failure, production can be switched over immediately to the remote site. The primary site's recovery time is irrelevant because production continues on the secondary systems. This approach provides better business continuance and results in significant savings because backup and recovery system costs are lower.
When not running production operations as part of a business continuance effort, the remote replicated location can be used for testing, development, and other functions. Oracle has developed the ability to generate reports from a copy of the secondary site's production database, enabling IT staff to offload work from the production system and keep the secondary database current. This method provides savings for backup and recovery and maximizes system availability. Many organizations distribute their recovery systems in this manner, with applications that can fail over from city to city. In fact, this design can be used almost universally, with the exception of firms that are required to have large and elaborate backup and recovery systems for compliance reasons.
Figure 3. Oracle technologies enable maximum protection plus cost savings.
Sites that employ Oracle's Zero Data Loss Recovery Appliance for database backups can benefit from the secure replication features of the recovery appliance. Backups on a local recovery appliance can be easily and quickly replicated via secure transport to a remote recovery appliance. Recovery operations can be run directly from the remote recovery appliance, without first staging the data locally.
Conclusion
When deploying Oracle engineered systems, it is vital to update backup and recovery platforms at the same time, because the resulting production environment will exhibit superior performance with far greater data throughput and scalability. Backup and recovery solutions that are not designed to handle the unique performance characteristics of Oracle engineered systems are unable to keep pace and likely will become the limiting factor in protecting against data loss and maintaining suitable service levels.
Oracle Optimized Solution for Secure Backup and Recovery, powered by Oracle servers, provides an excellent backup solution for heterogeneous networks. As a comprehensive end-to-end data archive solution with centralized management for simplified administration, Oracle Optimized Solution for Secure Backup and Recovery provides a scalable architecture that easily adapts to growing storage demands. Indeed, the high performance and capacity of this solution facilitates the consolidation and migration of existing SAN backups to more cost-efficient network backups.
With a highly flexible architecture, Oracle Optimized Solution for Secure Backup and Recovery features virtually unlimited scalability. The architecture supports multiple media servers working together under the administrative control of an administrative server. Additional media servers can be deployed or multiple backup domains can be configured to address evolving scalability requirements. Organizations using Oracle Optimized Solution for Secure Backup and Recovery can implement cost-effective and powerful backup solutions for the largest, most diverse data center environments.
For configuration information, recommendations, sizing information, and all the latest information on Oracle Optimized Solution for Secure Backup and Recovery, please refer to My Oracle Support Note 1558851.1.
See Also
For additional information, please see the following:
About the Author
Dean Halbeisen is a solutions manager at Oracle. He has over 20 years of IT experience and is an expert in enterprise computing solutions, most recently applying these practices to next-generation data center solutions, integrated systems, and Oracle engineered systems. In his current role, he is responsible for solution architecture and development around Oracle Optimized Solutions, including communicating about Oracle's systems, solutions, and technology strategies and roadmaps to customers, partners, and internal stakeholders.