While achieving CopyStorm performance can vary from company to company, the principles and practices in this article should help you optimize CopyStorm for your organization.

In most cases a company will want to optimize the rate at which records are backed up from Salesforce. In this case there are typically only a few parameters that typically yield performance improvements which:

• Increase the rate at which data arrives from Salesforce.
• Increase the rate at which your database can accept changes.

The best way to improve the rate at which data arrives from Salesforce is to:

• Increase the number of tables CopyStorm backs up at the same time. This value is controlled by a parameter called “# Salesforce Threads” under “Salesforce API” on the “Configuration” tab. Most companies will see optimal performance using a value of 3 to 10. The optimal value for your company will likely depend on the rate at which your database will accept data.

People often ask “Why not use the BULK API since it is the fastest approach?” The short answer is:

• Though the execution time reported by the BULK API is short, the amount of time a human has to wait (wall time) for a copy job to complete is longer than using multiple SOAP or REST threads. Ten SOAP threads will beat ten BULK jobs almost 100% of the time. We suspect that this is because BULK jobs run at a much lower priority in Salesforce than SOAP or REST requests.

There are two common cases where memory usage is a problem:

Large Attachments

Because the Salesforce SOAP API returns Attachments inline the memory requirements for large SOAP attachments will be almost four times the size of the attachment reported by Salesforce. There are two primary ways to solve this problem:

1. Increase the memory allowed to CopyStorm.
2. If using Oracle or SQL/Server switch to the REST API. In this case large Attachments will no longer be kept in memory and will be streamed directly to the database. The REST API helps slightly for other databases but not dramatically, as they require large objects to be kept in memory

Old Equipment

If you are forced to work on equipment that does not have much memory (e.g. an old 32-bit 2GB Windows box) then:

1. Huge attachments may not be possible (unless using Oracle or SQL/Server).
2. CopyStorm memory use can be limited at the cost of more Salesforce API calls by reducing the “Salesforce Batch Size” parameter.

CopyStorm typically has very modest compute requirements — if you find yourself in a situation requiring more memory or compute than available Capstorm generally recommends upgrading hardware (a standard desktop has more than enough resources to run CopyStorm).

If your CopyStorm database is heavily used for business analytics, operational reporting, or other purposes while a CopyStorm backup is running you may experience database performance issues due to lock contentions. The easiest way to solve this problem is to reduce the number of records sent to the database in a single transaction: