CopyStorm/SanityChecker

As of CopyStorm/Medic 10.54.2, the Base64 Sanity Checker function has been merged into CopyStorm/Medic.

For the latest documentation, please see:


When the Salesforce REST API returns a record with a BASE64 column, the BASE64 data is returned as a hyperlink that can be used to read the raw bytes.
Experience has shown that is is possible for the hyperlink returned by Salesforce to return a HTTP 404 (not found) or an HTTP 500 (internal lightning platform error) rather than the expected HTTP 200 (OK). When this happens:
  • The BASE64 data cannot be read by the REST API.
  • The same operation using the SOAP API results in an UNKNOWN_EXCEPTION or an unparsable SOAP body

The CopyStorm/SanityChecker tool tests the hyperlinks returned by the REST API and reports the ID of records where the hyperlink fails (e.g. Does not return a HTTP 200). It is often wise to check each reported record using the Salesforce GUI to make sure that the data is really available. If you find a corrupted Attachment record report it to Salesforce.

Step 1: Enter Salesforce Credentials

Step 2: Set Search Parameters

The only required parameters are the tables to search, but there are a number of other parameters you may want to modify.

Parameter Description
Modified After Search for records starting at this date. A date can be selected from the menu or typed in directly. Values are restricted to datetime formats supported by SOQL.
Modified Before Search for records up to this date. A date can be selected from the menu or typed in directly. Values are restricted to datetime formats supported by SOQL.
Salesforce Batch Number of records to process from Salesforce in a single batch. The default usually provides optimal performance.
# Checker Threads Number of base 64 hyperlinks to analyze concurrently.
Max Timeout (sec) Number seconds to allow for each hyperlink test. If a hyperlink HTTP request times out it is assumed to be OK based on Capstorm’s experience with problem records. Setting this to a smaller value may improve the overall throughput of the analysis.
Report Timeouts If this option is checked then all HTTP timeouts will be reported as potential problems. Enabling this feature may be useful in selecting an optimal Max Timeout value for your organization.

Step 3: Check Hyperlinks

The next step is to click on the Start Sanity Check button and wait for the analysis to complete. Note:

  • Each hyperlink check takes one Salesforce API call. There is one additional API call for every 2000 records.
  • An running analysis can be cancelled but the cancellation process may not be immediate. The reason is that pending HTTP requests must complete or timeout before the cancellation process is finished.

While the sanity check is in process, the tool will look like the following:

The following example shows what a report of potential problems looks like. You can copy/paste the potential problems into your favorite editor.

Summary

Hopefully when you scan your Salesforce the tool will not find potential problems. However, if potential problems are found then it is wise to check the validity of each record using the Salesforce GUI.

If you are using CopyStorm to perform backups, then this class of error will be discovered during the normal course of a backup. Unfortunately the error is often masked by an UNKNOWN_EXCEPTION or an Invalid SOAP Envelope response from Salesforce.