File and Folder Backup retention policies

At the last step of the File and Folder Backup wizard, there are new retention policies added.

These policies provide data deletion from the cloud of files that are either old (not changed on the local machine for X days) or do not exist on the machine.

The policy does not touch the data on the local machine. Peculiarities of the policies:

  • can be switched on separately or combined
  • reflect “natural” retention policies for old and unnecessary (deleted) files and can be used when a customer wants to save only recent versions of files in the cloud;
  • if a file has versions, the policies remove “all or nothing”;
  • the policies can be switched only when option “send email reports” is chosen. This is done for safety so customers always know that policies are switched on and what amount of data is deleted from the cloud.

How does it work?

  • Not changed on the local computer for X days

    After file uploading is completed, the next step of OBRM backup session is scanning all the metadata on backed up files in Protect_.db. If modifyDate field for some file corresponds to the condition not changed for ($X - 3) days, it fills the field notificationUnchangedSentOn with the date when this condition found. The field notificationUnchangedSentOn means, that at the end of the backup job, the name of this file will be included into the special email report (but not more than 50 names totally), which will be sent to the email address, mentioned in Backup Reports section. This report alerts the customer, that these files (and all their versions) will be deleted from the cloud in 3 days (or later, if the backup job is scheduled for a later time) during the next scheduled backup job. During the next backup job, the client checks whether these 3 days are already passed, and if it is true - deletes the file (and all its versions) from the cloud. If during these 3 days (or more, if the backup job is scheduled for a later time) the file is modified, the field notificationUnchangedSentOn is cleared.

  • Deleted from the local computer, after waiting X days

    After file uploading is completed, the next step of OBRM backup session is scanning all the metadata on backed up files in Protect_.db. If the application finds that some files are deleted from the local computer, it fills the field notificationDeletedSentOn with the date when this deletion found (not with the date when a file was in fact deleted). The field notificationDeletedSentOn means, that at the end of the backup job, the name of this file will be included in the special email report (but not more than 50 names totally), which will be send to the email address, mentioned in Backup reports section. This report alerts the customer that these files (and all their versions) will be deleted from the cloud in 3 days (or later, if the backup job is scheduled for later time) during the next scheduled backup job. During the next backup job, the client checks whether 3 days are already passed, and if this is true–deletes the file (and all its versions) from the cloud. If during these 3 days (or more, if the backup job scheduled for later time) the file is added to the local computer, the field notificationDeletedSentOn is cleared.

    notification_important If the retention policy is modified, all the fields notificationDeletedSentOn and notificationUnchangedSentOn for all the files cleared, and no one file is deleted until the updated retention policy conditions are met.

    The retention policy will not work for files which (for some reason) have no records in Protect_.db file. For example, the files that had been uploaded to the cloud and then deleted from the local computer. Then OBRM application was uninstalled (Protect_.db file was deleted) and the application was installed again. So, the application has no information about uploaded data to the cloud, which was deleted from the local computer.

    Example: Delete files from the cloud 5 days after they were deleted locally

    If you delete a file from the local computer, which is selected in the backup set, OBRM will wait for 5 days and then delete the same from the cloud.

    What one would like to know if those files deleted locally prior to the option being selected will be deleted from the cloud once 5 days have elapsed.

    Monday, July 18 – Delete File No.1

    Wednesday, July 20 – Upgrade OBRM to the latest version and select the option to delete files in the cloud 5 days after they have been deleted locally

    Thursday, July 21 – Delete File No.2

    It is clear that on Tuesday, July 26, File No.2 will be removed from the cloud, as that is 5 days after a local deletion that took place after the latest release of OBRM had been installed and the option was selected.

    The question is whether on Saturday, July 23, File No.1 will be deleted from the cloud, as that deletion occurred prior to the latest release of OBRM being installed and the option being selected? Is OBRM able to detect files delete prior to this release and option?

    File No.1 will be deleted on Thursday, July 25, because the first backup with the retention policy switched on will check whether files are presented on the disk. For the deleted files, it will record Deleted Date as the current backup date (when OBRM found out that files are not presented). According to the example, it will mark it as 07/20 and will delete them in 5 days.

    Explanation

    At the end of every backup session, OBRM checks whether files presented in the backup set have been deleted from the local machine.

    Please, note that the policy is not started by itself, only at the end of the backup session.

    Consider that the backups run daily from the machine, so the scenario is following:

    • Monday, July 18 – Delete File No.1

    Policy does not exist, no files are deleted or marked for deletion.

    • Wednesday, July 20 – Upgrade OBRM to the latest version and select the option to delete files from the cloud 5 days after local deletion:
    • Run backup
    • OBRM checks whether files from the backup set are presented on the disk
    • For files deleted before July 20 (i.e., File No.1 was deleted on July 18), OBRM sets Deleted Date as July 20
    • Email report is sent with the notification that files will be deleted in 5 days (07/25)

    • Thursday, July 21 – Delete File No.2
    • Run backup on July 20
    • File No.2 is marked for deletion on July 26

    • July 25
    • Run backup
    • File No.1 is deleted from the cloud (it was marked for deletion on July 20, when the retention policy was switched on)

Advanced Folder Backup retention policies

Two new retention policies were added:

  • Move Files To Cloud with X days to delay delete

    Corresponds to the older Time-Limited Backup. It deletes files at the chosen folder from the local machine if they were backed up more than X days ago (the last backup date is earlier than X days). If a file is modified within X days prior to the current backup session, it is backed up at the current backup session and the timer resets. Hence, if it is not be backed up within X days between the current backup session and the next backup session, it will be deleted from the local machine.

  • Temporary with several options

    Deletes files from the cloud (that are included to the chosen folder) that were not modified or not backed up within X days. A customer can choose between modification date and backup date.

Frequently asked questions

Would not the most recent backup date of a file be the same date as the most recent back up, in general?
In general, no. If a file is not changed, it is not backed up so the backup date remains older that the recent backup.
How does the file backup date different from the file modification date?
Modification date is a file attribute recorded in the filesystem. It is changed when a file is modified.

For example, the scheduled backup runs every day at 7 PM, and you modify a document every day at 5 PM, then, after the backup, the cloud will store the information:

  • file modification date – 5 PM
  • backup date – 7 PM

If a file is backed up once a week (i.e. on Sundays), then the backup date will differ from the modified date.

When is modification date of a file more recent than the backup date?
Modification date is changed if a file is modified. However, the contents of the file may not be changed.

If OBRM sees that the modification date of a file, it differs from the one recorded at the previous backup session, OBRM then calculates a hash of a file. If the hash is the same as before, the contents of a file is actually not changed so it is not uploaded to the cloud. So, the backup date remains an old one, but modification date appears to be more recent.

How many days is the limit on retention policies for backup software? Is it 365 Max?
The limit of days for retention policies is 3660 days.

Include sub-folders

Advanced Folder Backup operates on the whole folders added:

  • no options to choose a single file or several files when configuring Advanced Folder Backup
  • no options to EXCLUDE the subfolders when a large parent folder is added (then, one needs to Add the needed subfolders to the wizard separately

Hence, Include sub-folders was implemented as an option for the customers who want to choose whether the retention policy should check its subfolders or operate only on files that are placed directly under the folder.

info Include sub-folders option determines the choice where to remove, not what to backup.

Example: The following retention policy will backup all files and folders from E:\downloads, but it will delete from the cloud the files which are modified earlier than 10 days ago and are stored under the folder E:\downloads directly. Any files from its subfolders will not be deleted from the cloud.

Delete file versions

It deletes all file versions, which modification date or backup date older than X days.

However, as OBRM has cache options, it stores baselines and deltas (differentials from the last full backup of the file).

The versions’ deletion rule is the following: Baseline (B) delta (d) | Threshold (older than X days).

B1 B2 | B3 B4 => B3 B4
B1 d11 d12 | d13 B2 d21 d22 => B1 d13 B2 d21 d22
B1 d11 d12 | B2 d21 d22 => B2 d21 d22
B1 d11 | B2 => B2
B1 d11 d12 d13 | => remove all versions of the file

Additional settings

Skip files modified more than X days ago
If this option is enabled, the files in the added folder/subfolders which were modified more than X days ago, will not be added to the backup set and backed up.