How Do I Backup Massive Salesforce Attachments?

If a Salesforce instance contains extremely large Attachments (600MB – 2000MB), backing them up with CopyStorm may take a special approach. The reasons are rooted in the fact that none of the APIs provided by Salesforce provide a sufficient solution “out of the box”.

• Fact 1: The Salesforce SOAP API returns attachment bodies as part of the SOAP XML envelope. This can cause two problems:
  • The SOAP envelope will cause large memory requirements for the XML parser. Increasing the maximum amount of memory allowed for CopyStorm will usually solve this problem.
  • If the SOAP envelope is too large to fit in a single string variable, the XML parser will suffer a fatal error. The largest string the XML parser can handle is 2GB and the string in a SOAP envelope can exceed this length.
• Fact 2: The Salesforce REST API returns attachment bodies as hyperlinks to the body content. This approach minimizes the memory requirements for CopyStorm but has some downsides:
  • Salesforce throttles the retrieval of attachment bodies to 1MB per second AND limits the time of retrieval to 10 minutes. This means that attachments larger than roughly 600MB cannot be retrieved using the REST API under default Salesforce parameters.

There is a way to get around this issue but it requires a call to Salesforce support:

• Call Salesforce and ask them to increase your Organization’s maximum timeout to a value that will allow your largest attachment to be read (at a rate of ~1MB per second).
• In CopyStorm, backup massive Salesforce Attachments using the REST API.
  • The default is SOAP — it is generally faster.