Full vs. incremental vs. differential backup
Incremental, differential, and full backups are common techniques. Less frequently encountered types include forever-incremental, synthetic, and mirror backups.
Full backups: Full backups are complete copies of all configured data. This backup is best used periodically, although it is essential to have all data entirely backed up, because creating and implementing a full backup regularly consumes far more storage, time, network bandwidth, and other resources.
Incremental backups: Incremental backups save resources and time because they back up only the data that changed since the last backup of any kind. This consumes less time and storage space but makes restoration more difficult as it means restoring both the last incremental backup and the last full backup as well.
A full backup and a level 0 incremental backup are physically identical, with one difference: the level 0 backup can be used as the parent for a level 1 backup because it is saved in the RMAN repository, an online backup source of read/write datafiles, as an incremental backup.
Differential backups: Differential backups, also called cumulative incremental backups, also save resources and time because they backup only the data that changed since the last full backup — but in this case, where the incremental backup goes back to the last backup of any type, the differential backup restores data back to the last full backup only.
Comparing full, incremental and differential backups, the full backup is just like it sounds; the incremental backup covers just the most recent changes from the last backup of any type; and the differential backup goes back to the last full backup.
Backup systems perform full backups in many different ways. An incremental-forever approach performs just one full backup ever, followed by incremental backups. Block-level incremental-forever backups, a subset of this type, store the backup on the changed blocks rather than all files that have been created or modified since the last backup.
The synthetic full backup process can also reconstruct a full backup by copying data from existing backups. A synthetic full backup uses only the backup server and its storage; it consumes no resources of the system it is backing up.
Similar to the full backup, a mirror backup creates an exact copy of the source data set. However, while a classic full backup will track different versions of the files, in a mirror backup, only the latest data version is stored in the backup repository; the mirror backup copies only modified files. All individual backup files are stored separately, just as they are in the source, not in a single compressed/encrypted container file.
Mirror backups allow for quick, direct access to individual backup files without a formal restore operation. However, they also demand large amounts of storage, and present a high risk of data loss, data corruption, misuse, and unauthorized access. Furthermore, mirror backup files are actual mirrors, meaning they reflect what happens in the source files. Any adverse modifications to the source due to accident, human error, malware action, or sabotage may produce the same outcome in the mirror backup.
Derrick Rountree, in
Security for Microsoft Windows System Administrators, 2011 Differential backups are usually used in
combination with full backups. Differential backups will only back up all files that have the archive bit set. Because of this they will take a shorter amount of time to perform than full backups or copy backups. Differential backups do not reset the archive bit. So basically, every time you perform a differential backup you will be backing up every file that changed since the last full backup was performed. Complete restores using differential backups will generally take longer to perform than
full backup restores because you will have to restore both the last full backup and the last differential backup.Organizational and Operational Security
Differential Backups
Read full chapter
URL: //www.sciencedirect.com/science/article/pii/B9781597495943000053
MCSE 70-293: Planning, Implementing, and Maintaining a High-Availability Strategy
Martin Grasdal, ... Dr.Thomas W. ShinderTechnical Editor, in MCSE [Exam 70-293] Study Guide, 2003
Differential Backups
The differential backup type is sometimes used as a substitute for the incremental type. A differential backup collects data that has changed or been created since the last full [normal] or incremental backup, but it does not clear the archive bit on the file. It can also be used after a copy or differential backup, but as with an incremental backup, every file with the archive attribute set is backed up.
The differential backup is advantageous when you want to minimize the restoration time. A complete system restore with a full/differential backup combination, as illustrated in Figure 8.36, requires only the most recent full backup and the most recent differential backup. Differential backups start with small volumes of data after a recent full or incremental backup, but often grow in size each time, because the volume of changed data grows. This means that the time to perform a differential backup starts small but increases over time as well. In theory, if full or incremental backups are infrequent, a differential backup could end up taking as long and reaching the same volume as a full backup.
Figure 8.36. Full [Normal] Backup/Differential Backup Pattern
You may also want to use combinations of full [normal], incremental, and differential backups. For instance, if you begin with a full backup over the weekend, it might make sense to perform differential backups on Monday and Tuesday. By later in the week, the quantity of changes may be such that a differential backup cannot be performed overnight. An incremental backup on Wednesday will likely solve the problem, with differential backups continuing after that. Using this system, the restore times are still minimized, because the maximum restoration would involve tapes from the full, incremental, and one differential backup. If a failure occurred before Wednesday, it may take tapes from only the full and, possibly, a differential backup to restore the system.
Read full chapter
URL: //www.sciencedirect.com/science/article/pii/B9781931836937500129
Domain 7: Security Operations [e.g., Foundational Concepts, Investigations, Incident Management, Disaster Recovery]
Eric Conrad, ... Joshua Feldman, in CISSP Study Guide [Third Edition], 2016
Differential
Another approach to data backup is the differential backup method. While the incremental backup only archived those files that had changed since any backup, the differential method will back up any files that have been changed since the last full backup. The following is an example of a backup schedule using tapes, with weekly full backups on Sunday night and daily differential backups.
Each Sunday, a full backup is performed. For Monday’s differential backup, only those files that have been changed since Sunday’s backup will be archived. On Tuesday, again those files that have been changed since Sunday’s full backup, including those backed up with Monday’s differential, will be archived. Wednesday, Thursday, Friday, and Saturday would all simply archive all files that had changed since the previous full backup.
Given this schedule, if a data or disk failure occurs and there is a need for recovery, then only the most recent full backup and most recent differential backup are required to initiate a full recovery. Though the time to perform each differential backup is shorter than a full backup, as more time passes since the last full backup the length of time to perform a differential backup will also increase. If much of the data being backed up regularly changes or the time between full backups is long, then the length of time for a backup might approach that of the full backup.
Read full chapter
URL: //www.sciencedirect.com/science/article/pii/B9780128024379000084
Domain 7: Operations Security
Eric Conrad, ... Joshua Feldman, in Eleventh Hour CISSP [Second Edition], 2014
Incremental and differential
Incremental backups only archive files that have changed since the last backup of any kind was performed. Differential backups will archive any files that have been changed since the last full backup.
Did You Know?
Assume a full backup is performed every Sunday, and either incremental or differential backups are performed daily from Monday to Saturday. Data is lost after Wednesday’s backup.
If incremental daily backups were used in addition to the weekly full backup, the tapes from Sunday, Monday, Tuesday, and Wednesday would be needed to recover all archived data.
If differential backups were used in addition to the full weekly backup, only the Sunday and Wednesday tapes would be needed.
Read full chapter
URL: //www.sciencedirect.com/science/article/pii/B9780124171428000078
Operational Activities
Kelly C. Bourne, in Application Administrators Handbook, 2014
14.1.1 Creating backups
The importance of successfully creating backups can’t be overemphasized. It’s easy to overlook the backup process once it is set up and running smoothly, but that’s a huge mistake. You need to regularly confirm that the backup process is error-free, that the output is actually useable. You also need to periodically confirm that you’re backing up everything that you think you are.
Most applications have a backup created every day. There are however different types of backups that can be taken of an application. Some different types are:
•Full backup—every file is backed up. Usually, this is done weekly.
•Differential—backs up the files that have been changed since the last full backup.
•Incremental—only backs up the files that have been changed since the last incremental or full backup.
14.1.1.1 What is being backed up?
Creating backups on a regular basis is a fundamental necessity, but do you know exactly what is being backed up? There are files that need to be backed up and others that don’t need to be preserved. Are the correct files being backed up? Some examples of what should be backed up so that the server could be restored if the drives failed or became corrupt include the following file types:
•Data
•Executables
•Customizations your organization put into place
•Support files like the registry, ini files, resources files, properties files, xml files, etc.
•User profiles
•Log files
14.1.1.2 Backup file storage
Once backups are created, then they need to be protected. Where are backup tapes or disks being stored? Is the stage site physically secure, i.e., does it have locked rooms, sign-in/sign-out procedures, etc.? Is it protected from the elements like heat, cold, and water?
It is absolutely necessary that your backup plan includes off-site storage of backup media. This is needed because if the backup media is stored in your production site and a disaster occurs, then you could lose everything! The off-site storage could be with a commercial operation like Iron Mountain or at another location that your organization operates. You need to ensure that the storage site is distant enough that it won’t be impacted by a disaster striking the production facility.
If backup media is stored off-site, then a number of additional questions need to be asked:
•What is the process for retrieving backup media from the other site?
•How quickly can it be delivered to the production or DR site?
•Where, i.e., on what drives, will it be loaded?
•Will the normal purging process of backup files delete the restored files? This could occur if your purging process is date driven, i.e., any files older than “x” days are automatically deleted.
•Has the process of requesting backup media and restoring it been tested? Has this been tested recently?
14.1.1.3 Retention
How long are your backups retained? Are critical backups, e.g., end of quarter or end of year, retained for longer periods of time? Is this retention period driven by a legal requirement for your industry?
What happens to a disk or tape once it is released?
•Are the files on it destroyed?
•If a physical media, e.g., tape, is involved, is it destroyed or reused?
•Is file destruction done in such a way it’s impossible for files to be recovered?
•Does your industry have standards for how backup files are to be destroyed? Does your organization follow these standards?
14.1.1.4 Restoring from backups
Restoring from backups is a high-pressure activity. Not only is it done relatively infrequently, but most likely if it’s being done then it’s possible that something very bad has happened to the production environment. To maximize the chances of success, the restore process needs to be meticulously documented and regularly tested. The documentation on this process needs to be readily available to the team that will need it.
Restoring the system from backups depends on the type of backups that were made. Restoring from a full backup is strictly a matter of loading the files from backup media. If the restore is being done from a combination of full and differential or incremental files, then you need to load the most recent full backup media and then the differential or incremental media in chronological order. For example, if full backups are made every Friday and differential backups are made on other days, then the steps to restore the system on Wednesday would be:
1.Load the Friday full backup files
2.Load the Saturday differential files
3.Load the Sunday differential files
4.Load the Monday differential files
5.Load the Tuesday differential files
Doing a full backup every night would be more straightforward, but they take longer and consume significantly more storage. Combining full backups with incremental or differential backups is more efficient and requires less space, but complicates the recovery process.
Some questions that need to be thought out in advance include the following:
•Who is authorized to do decide that a restoration should take place?
•Does this person have an alternate in case the primary decision maker isn’t available?
•Has the restoration process been practiced? When was the last time it was done?
•How long does it take to perform a restore?
14.1.1.5 Encryption
If your data contains PII [Personally Identifiable Information], then the backup media should be encrypted. This is especially true if it is being stored off-site. There are regular stories in the trade press about backup tapes that get misplaced. If they aren’t encrypted, then any data on them is readily available to whoever found, or stole, the media.
What form of encryption is being used? Is it strong enough to keep the bad guys at bay? The field of security and encryption is changing on a daily basis. Any guidelines that I could write today will be outdated before this book hits the shelf. Your best course of action is to consult with experts regarding what encryption algorithm and key size should be chosen and deployed.
Read full chapter
URL: //www.sciencedirect.com/science/article/pii/B9780123985453000145
SQL Server Backup and Recovery
In Designing SQL Server 2000 Databases, 2001
Differential Backup
Between two complete backups, segments of data in the database, called pages, will be changed and new data will be written. Instead of making complete backups over short intervals, you can choose to back up only the changed pages—the differences between the complete backup and the current state of the database. This is called a differential backup and holds all changed pages since the last complete backup, even if another differential backup has been made. This type of backup enables you to roll back the database to a certain point in the past by restoring the complete backup and one appropriate differential backup.
In the case of a large database, you will keep making differential backups, up to the moment the backup size becomes too large or can no longer finish within a certain amount of time. For example, say that you have a 100GB database and the differential backup is 50GB, and it takes about 4 hours to complete. Both values could be the cue to make another complete backup. Especially in the case of a restore, you must restore 150GB for a 100GB database, costing valuable time.
Since the database will have changed since the last differential backup, you will also have to restore the transaction log after the differential backup to return the database to its most recent state.
Read full chapter
URL: //www.sciencedirect.com/science/article/pii/B9781928994190500105
Locking Down Your XenApp Server
Tariq Bin Azad, in Securing Citrix Presentation Server in the Enterprise, 2008
Understanding Backup Types
The types of backups you can choose in most commercial backup programs as well as the backup utility provided with Windows 2003 are as follows:
▪Normal
▪Incremental
▪Differential
▪Copy
▪Daily
As you will see in the paragraphs that follow, different types of backups archive data in different ways. Because of this, the methods used to back up data will vary between businesses. One company might make normal backups every day, while another might use a combination of backup types. Regardless of the types used, however, it is important that data be backed up on a daily basis so that large amounts of data won't be lost in the event of a disaster.
Note
Some types of data require that you follow special procedures to back them up. The System State data, discussed in the text, is one such special situation.
Another special situation occurs when you want to back up files that are associated with Windows Media Services [WMS]. To back up these files, you must follow the procedures that are outlined in the WMS Help files. You cannot use the normal backup procedures to back up and restore these files.
Microsoft recommends that if you want to back up database files on an SQL server, you should use the backup and restore utilities that are included with SQL Server instead of the Windows Server 2003 Backup utility.
If your Windows Server 2003 computer is running cluster services [Enterprise or Datacenter editions], you need to perform an ASR backup for each cluster node, back up the cluster disks in each node, and then back up individual applications that run on the nodes.
Before describing each of the backup types, it is important to understand that the type chosen will affect how the archive attribute is handled. The archive attribute is a property of a file or folder that's used to indicate whether a file has changed since the last time it was backed up. As you will see in the paragraphs that follow, depending on the backup type used, the archive attribute of a file is or is not cleared after it is backed up. When the file is modified, the archive attribute is set again to indicate it has changed and needs to be backed up again. Without the archive attribute, your backup program is unable to tell whether files need to be backed up or not. Here is a description of each backup type in more detail:
▪Full Backup The full backup, as its name implies, backs up everything specified by the user performing the backup operation. A full backup can include the operating system, system state data, applications, and any other data. With a full backup, everything that is backed up has the file system archive bit reset [cleared].This allows the incremental and differential backup types to determine if the file needs to be backed up. If the bit is still clear, the other backup types know that the data has not changed. If the bit is set, the data has changed, and the file needs to be backed up. The full backup is usually the first backup performed on a server. It takes the longest of all the backup types to complete, because it backs up all specified files, regardless of the state of the archive attribute. A full backup consumes the largest amount of backup media of any backup type. Depending on the amount of information chosen to back up and the underlying backup technology involved, it may require multiple backup media to complete. The main advantage of the full backup type is the ability to rapidly restore the data. All of the information is contained in a single backup set when this type of backup is used. The disadvantages of full backups are high media consumption and long backup times.
▪Incremental Backups During an incremental backup operation, all specified files have their archive bit examined. If the bit is set, the file is backed up, and then the bit is cleared. This backup type is used to back up data that has changed or been created since the last full [normal] or incremental backup. It can also be used after a copy or differential backup, but because these do not reset the archive attribute, there is no way for the incremental backup to tell which files have changed since one of those backups last ran. As a result, every file with the archive attribute set is backed up. The incremental backup type is used between full backups. It is quick to perform, collects the least amount of data, and consumes the smallest amount of media. A complete restore, however, requires the last full backup and every incremental backup [in sequence] since the full backup was performed. The primary benefits of using the full/incremental backup combination are time and media savings. The main drawback of this combination is longer and more complex restore operations if there are long periods between full backups.
▪Differential Backups The differential backup type is sometimes used as a substitute for the incremental type. A differential backup collects data that has changed or been created since the last full [normal] or incremental backup, but it does not clear the archive bit on the file. It can also be used after a copy or differential backup, but as with an incremental backup, every file with the archive attribute set is backed up. The differential backup is advantageous when you want to minimize the restoration time. A complete system restore with a full/differential backup combination requires only the most recent full backup and the most recent differential backup. Differential backups start with small volumes of data after a recent full or incremental backup, but often grow in size each time, because the volume of changed data grows. This means that the time to perform a differential backup starts small but increases over time as well. In theory, if full or incremental backups are infrequent, a differential backup could end up taking as long and reaching the same volume as a full backup.
▪Volume Shadow Copy More of a Windows 2003 feature than a backup type, Volume Shadow Copy allows you to back up all files on the system, including files that are open by applications or processes. In previous versions of Windows, the applications would need to be stopped or users logged out to allow these files to be closed and backed up using a backup program. With Volume Shadow Copy, these files can continue to remain in use without affecting the integrity of the backup. This feature is enabled by default, but it may need to be disabled if data managed by some critical applications would be affected by the use of Volume Shadow Copy
Warning
Not all backups are the same. Remember that normal and incremental backups set the archive attribute after backing up a file, but differential, copy, and daily backups do not. You can use normal, copy, and daily backups to restore files from a single backup job, whereas you can use incremental and differential backup types in conjunction with normal backups. This is because differential backups back up all files that have changed since the last normal backup [regardless of whether they were backed up by a previous differential backup], while incremental backups only back up files that have changed since the last normal or incremental backup and were not backed up previously.
Read full chapter
URL: //www.sciencedirect.com/science/article/pii/B978159749281200007X
Backup
Pierre Bijaoui, Juergen Hasslauer, in Designing Storage for Exchange 2007 SP1, 2008
Backup Types and Schedule
Four backup types are available: full, copy, incremental, and differential. Full backups create a complete copy of your database on the backup media. Upon successful completion of the backup, the transaction log files are purged. Copy backups also create a complete copy of the data, but a copy backup does not purge the log files or stamp the database header with the date and time of the backup. Copy backups are used to duplicate a database for testing purposes. They are not used within a regular backup strategy.
Incremental and differential backups always reference the last full backup. Both backup types only copy the existing transaction log files to the backup media; they do not copy the Exchange database to the backup media. These backup types enable a shorter backup window because less data are written to the backup media. Incremental backups purge existing log files at the end of a backup. You have to remember that in case of a restore, you need enough disk space on your log file disk volume for all transaction log files generated since your last full backup. Differential backups do not delete transaction log files. With differential backups, log files accumulate until you run the next full backup.
You should select a backup type and schedule according to your service level requirements [Table 8-2] and the RPO and RTO.
Table 8-2. Full/differential/incremental backups
| Recovery steps | Last backup set | Last full and last differential backup set. | Last full and all incremental backup sets since the full backup. |
| Recovery issues | None | Many log files have to be replayed. | Many individual backup sets have to be restored, and many log files have to be replayed. If one incremental backup set is damaged, you can only restore to the last contiguous log file/backup set. |
| Recovery time | Low | Medium to high | High |
| Backup time | High | Medium | Low |
| Media costs | High | Medium | Low |
Using daily full backups is a best practice. Even if you have a shorter backup window with the other backup types in Table 8-2, the recovery procedure is more complex and time consuming. Remember that you do not back up for the sake of backups. You only perform backups to be prepared for a fast recovery.
If you made the decision to have large databases because business needs justified the support for mailboxes with gigabyte quotas, you will face the issue that you cannot back up all databases using a daily full backup. Very likely your backup infrastructure is not capable of transferring terabytes of data from the mailbox server to a backup media within the backup window. In this case, you can consider using weekly full and daily differential backups. Table 8-3 shows the corresponding backup schedule.
Table 8-3. Weekly full and daily differential
| 1 | Full | Dif | Dif | Dif | Dif | Dif | Dif |
| 2 | Dif | Full | Dif | Dif | Dif | Dif | Dif |
| 3 | Dif | Dif | Full | Dif | Dif | Dif | Dif |
| 4 | Dif | Dif | Dif | Full | Dif | Dif | Dif |
| 5 | Dif | Dif | Dif | Dif | Full | Dif | Dif |
| 6 | Dif | Dif | Dif | Dif | Dif | Full | Dif |
| 7 | Dif | Dif | Dif | Dif | Dif | Dif | Full |
| … | … | … | … | … | … | … | … |
Read full chapter
URL: //www.sciencedirect.com/science/article/pii/B9781555583088000089
Domain 6
Eric Conrad, ... Joshua Feldman, in CISSP Study Guide, 2010
Electronic Backups
Electronic backups are archives that are stored electronically and can be retrieved in case of a disruptive event or disaster. Choosing the correct data backup strategy is dependent upon how users store data, the availability of resources and connectivity, and what the ultimate recovery goal is for the organization.
Full Backups
A full system backup means that every piece of data is copied and stored on the backup repository. Conducting a full backup is time consuming, bandwidth intensive, and resource intensive. However, full backups will ensure that any necessary data is assured.
Incremental Backups
Incremental backups archive data that have changed since the last full or incremental backup. For example, an example site performs a full backup every Sunday, and daily incremental backups from Monday through Saturday. If data are lost after the Wednesday incremental backup, four tapes are required for restoration: the Sunday full backup, as well as the Monday, Tuesday, and Wednesday incremental backups.
Differential Backups
Differential backups operate in a similar manner as the incremental backups except for one key difference. Differential backups archive data that have changed since the last full backup.
For example, the same site in our previous example switches to differential backups. They later lose data after the Wednesday differential backup. Now only two tapes are required for restoration: the Sunday full backup and the Wednesday differential backup.
Electronic vaulting
Electronic vaulting is the batch process of electronically transmitting data that is to be backed up on a routine, regularly scheduled time interval. It is used to transfer bulk information to an offsite facility. There are a number of commercially available tools and services that can perform electronic vaulting for an organization. Electronic Vaulting is a good tool for data that need to be backed up on a daily or possibly even hourly rate. It solves two problems at the same time. It stores sensitive data offsite and it can perform the backup at very short intervals to ensure that the most recent data is backed up.
Because electronic vaulting occurs across the Internet in most cases, it is important that the information sent for backup be sent via a secure communication channel and protected through a strong encryption protocol.
Remote Journaling
A database journal contains a log of all database transactions. Journals may be used to recover from a database failure. Assume a database checkpoint [snapshot] is saved every hour. If the database loses integrity 20 minutes after a checkpoint, it may be recovered by reverting to the checkpoint, and then applying all subsequent transactions described by the database journal.
Remote Journaling saves the database checkpoints and database journal to a remote site. In the event of failure at the primary site, the database may be recovered.
Database shadowing
Database shadowing uses two or more identical databases that are updated simultaneously. The shadow database[s] can exist locally, but it is best practice to host one shadow database offsite. The goal of database shadowing is to greatly reduce the recovery time for a database implementation. Database shadowing allows faster recovery when compared with remote journaling.
HA options
Increasingly, systems are being required to have effectively zero downtime, an MTD of zero. Recovery of data on tape is certainly ill equipped to meet these availability demands. What is required is the immediate availability of alternate systems should failures or disaster occur. A common way to achieve this level of uptime requirement is to employ a high availability cluster.
Note
Different vendors use different terms for the same principles of having a redundant system actively processing or available for processing in the event of a failure. Though the particular implementations might vary slightly, the overarching goal of continuous availability typically is met with similar though not identical methods, if not terms.
The goal of a high availability cluster is to decrease the recovery time of a system or network device so that the availability of the service is less impacted than would be by having to rebuild, reconfigure, or otherwise stand up a replacement system. Two typical deployment approaches exist:
•Active-active cluster involves multiple systems all of which are online and actively processing traffic or data. This configuration is also commonly referred to as load balancing, and is especially common with public facing systems such as Web server farms.
•Active-passive cluster involves devices or systems that are already in place, configured, powered on, and ready to begin processing network traffic should a failure occur on the primary system. Active-passive clusters are often designed such that any configuration changes made on the primary system or device are replicated to the standby system. Also, to expedite the recovery of the service, many failover cluster devices will automatically, with no required user interaction, have services begin being processed on the secondary system should a disruption impact the primary device. It can also be referred to as a hot spare, standby, failover cluster configuration.
Read full chapter
URL: //www.sciencedirect.com/science/article/pii/B978159749563900007X