Can Multiple CopyStorm Jobs Run on the Same Salesforce Backup Database?

CopyStorm supports the use of multiple running CopyStorm jobs to backup the same Salesforce instance to a Salesforce backup database, and there are some situations in which this can be preferable to a single job. For example:

  • A company has 20 CopyStorm Oracle tables that they want to be no more than 10 minutes behind the data in Salesforce. An incremental backup of all tables would take too long to allow a job to run every 10 minutes.
  • A company has a limited number of Salesforce API calls and does not want to accidentally use them all during the week. On the weekend they decide to backup the API-expensive tables (Attachment and ContentVersion).

While there is no practical limit to the number of CopyStorm jobs that can run at the same time while writing to the Salesforce backup database, there are a few restrictions:

  • Two CopyStorm jobs SHOULD NEVER be configured to backup the same table if they can run at the same time.
    • While this is unlikely to cause database issues, a lot of exceptions will likely be reported.
  • Salesforce will randomly kill a session if too many queries are active for an organization. This is a Salesforce governor limit and is typically around 20.
    • CopyStorm often generates longer running queries, so this has a material effect on how many CopyStorm jobs can be run at the same time on the same Salesforce organization.

One Common Good Practice

One common good practice is to divide Salesforce tables into two groups:

  • Group A: Tables where it is important to keep the data in your database in near real-time sync with Salesforce.
  • Group B: All tables not in Group A.

The configuration file for Group A lists the tables to be backed up in the “Included Tables” section.

The configuration file for Group B leaves the “Include Tables” section blank and puts the tables from “Group A” in the “Excluded Tables” section.

The CopyStorm job for Group A is scheduled to run every 30 to 60 minutes. The CopyStorm job for Group B is scheduled to run every night or every weekend.

Since the tables in Group A and Group B do not overlap, it is safe to run both jobs at the same time.