Restoring a Salesforce custom setting is a simple process once you understand:

• Each custom setting is kept in a Salesforce SObject where the sobject has a extra flag indicating that it is a Custom Setting.

A custom setting restore is a two step process.

Step 1: Restore the Custom Setting SObject

The first step is to restore the SObject.

• Goto the CopyStorm/Restore MetaData tab

• Search for the custom setting table in the list of CustomObjects.

• Restore the table.

After this step your target Salesforce will contain the custom setting but no values will be set for the setting.

This step restores the customer setting data using CopyStorm/Restore (just like any other table).

• Goto the Restore Set Editor Tab

• Select the table associated with the custom setting.

• Make sure the “All” record selected is active.

• Run the restore.

CopyStorm/Restore automatically restores deleted records for all record selectors except for “All” and “Random”. To restore deleted records with these selectors, check the “Include Deleted Records” checkbox.

In a CopyStorm backup a record is deleted if the “IsDeleted” column is set to true. There is nothing special about the “IsDeleted” column — it is a normal column Salesforce uses to indicate that a record has been deleted.

Example: Include Deleted Records in a “Select All” Record Selector

Example: Include Deleted Records in a “Random” Record Selector

Example: Only Include Deleted Records Using a Filter

CopyStorm/Restore uses a small footprint relational database (based on H2) to keep track of all restore activity. Like any database, it is possible (but rare) for this database to become corrupted. This article explains how to recover a corrupted H2 database.

Step 1: Create a New Directory and….

1. Copy the corrupted H2 database into the new directory.

2. Copy the file [CopyStormRestore Install Directory]/lib/h2*.jar into the new directory.

3. Open a command prompt and run java -cp h2*.jar org.h2.tools.Recover

4. The previous step will create a .txt file with the same name as your database. It may take a while to run and does not produce output.

java -cp h2*.jar org.h2.tools.RunScript -url "jdbc:h2:file:~/tmp/RecoveredTracker" -user root2 -password capstorm -script EMR-UAT-tracker.mv.txt -showResults

• The database in the -url parameter should not exist.

• The -user and -password parameter should NOT match a user in your corrupted database. root/capstorm are the defaults for a CopyStorm/Restore tracker database.

Of course, there is no guarantee that the recovery will work. Each case is likely to be unique. However, if the following tables are intact and have data there is a decent chance of recovery.

• RestoreConfig — this table must have values. It is used by CopyStorm/Restore to determine the version of the tracker database.

• RestoreTable — this table lists all Salesforce tables in the tracker and references to a unique tracking table for each Salesforce table. Do a SELECT * on this table.

• RestoreTableRecordReference

See this article for details about what is in a tracker database.

Before restoring Attachments with CopyStorm/Restore it may be useful to understand how Attachments are handled by Salesforce and how they are different from most tables. When a table in Salesforce contains a reference to another table it is almost always possible to know the type of table being referenced. For example, the Contact.AccountId field will always reference a record in the Account table. However, in the Attachment table, a number of reference fields are not so specific (e.g. Attachment.parentId) and can reference many built-in and all custom tables in Salesforce. This type of table is also known as “polymorphic” because they can “morph” into being a child of almost any type of table.

When restoring polymorphic table records, the best way to find the records is almost always to start with their parent table.

If this is a bit confusing, here is a specific example for restoring Attachments related to Accounts:

This informs CopyStorm/Restore to avoid inserting or updating Account records. The application will still find candidate records — this is necessary because the Attachments to be restored reference Accounts.

This informs CopyStorm/Restore that it should:

This article discusses common reasons that may prevent records to restore using CopyStorm/Restore.

Your Salesforce instance may have triggers that need to be disabled prior to restoring records. Here are a few examples:

• A BeforeUpdate trigger computes sales tax for a Quote. The error seen when restoring records is “TOO MANY SOQL STATEMENTS”.

• A BeforeInsert trigger does data validation on a specific field whose validation rules have changed over time. The error seen indicates that an INSERT or UPDATE failed.

Errors caused by triggers may be complex because a failure at a high level (e.g. Opportunity) may cause subsequent failures.

CopyStorm/Restore’s default configuration will not restore Opportunity or Quote records referencing inactive price books or prices. To include these records you must explicitly enable the behavior on the Global Parameters tab.

In Salesforce, it is common to see attachments and files associated with EmailMessage records. The Salesforce API however has restrictions about linking ContentDocumentLink records to EmailMessage records over the API when the EmailMessage record status is not ‘Draft’ and produces an exception in this scenario. This information be found in the Salesforce Object Reference for EmailMessage here.

On the EmailMessage object however, there is a field called IsClientManaged that when true allows users to modify EmailMessage.ContentDocumentId sto link file attachments even when the Status of the EmailMessage is not set to ‘Draft’. As of the 10.64.3 GA release of CopyStorm/Restore, an additional parameter in the Restore Set Editor for EmailMessage can be selected to push EmailMessage records with IsClientManaged = true regardless of the value in the CopyStorm database.

NOTE: This behavior is disabled by default and must be enabled through the CopyStorm/Restore configuration file.

How to Push EmailMessage Records with IsClientManaged = true

Follow the below steps to push EmailMessage records with the IsClientManaged field set to ‘true’ for all records:

• Open CopyStorm/Restore and navigate to the Restore Set Editor tab

• Click on EmailMessage in the lefthand list of objects

• In the Select Records section of the EmailMessage Restore Rules find the Additional EmailMessage Parameters section and check “Set IsClientManaged”

CopyStorm/Restore supports the ability to restore Salesforce data directly from a Snowflake database. This allows organizations that use CopyStorm to archive Salesforce data into Snowflake (for analytics or storage) to also leverage that same data for recovery purposes.

When configured with a Snowflake connection, CopyStorm/Restore behaves similarly to how it would with a traditional relational database. All standard restore operations are supported as long as the data exists in Snowflake.

When to Use CopyStorm/Restore with Snowflake

• You’re already archiving Salesforce data to Snowflake with CopyStorm.

• You need a simple recovery option without setting up a second relational database.

• You don’t need , , or file attachment support.

• Metadata Restore – Metadata cannot be restored from Snowflake.

• Point-in-Time Restore / Snapshots – Snapshot data is not supported in Snowflake.

• CS:Govern Restore – CS:Govern is not available for Snowflake.

• Base64 / Binary Content – Snowflake does not support native CLOB or binary types in the required format. As a result, objects like Attachment & ContentVersion cannot be restored from Snowflake.

By default, CopyStorm/Restore tracker databases are stored in $HOME/.capstorm/copyStormRestoreSets as . Documentation on the database schema and how to access it can be found here. You will also need the answer to the question:

• What Username and Password do I need to access a tracker database?

The answer is:

• Username: root

Though CopyStorm/Restore can faithfully restore Salesforce audit fields, there are a number of cases where the LastModifiedBy and LastModifiedDate fields cannot be restored. This article explains why.

If both the CreatedBy/CreatedDate and the LastModifiedBy/LastModifiedDate fields are not being restored accurately then the answer is simple. The Salesforce credentials in use are missing the necessary privileges to write to audit fields. See the article on Restoring Audit fields for instructions to enable this Salesforce feature.

If your Salesforce credentials do have the proper permissions, then several other conditions can cause the LastModifiedBy and LastModifiedDate fields to fail to match the values in the CopyStorm database. To understand the condition details, you must first understand what the option Salesforce permits on audit fields means. The permission that allows writing to Salesforce audit fields is called:

• Set Audit Fields upon Record Creation

The permission means exactly what it says: Audit fields can be set on a record INSERT but NOT on a record UPDATE. This “feature” is at the core of why CopyStorm/Restore cannot always restore the LastModifiedBy/LastModifiedDate fields to match what is in the backup.

The rest of this article describes scenarios where CopyStorm/Restore is forced to UPDATE records and thus change their LastModifiedBy and LastModifiedDate fields.

Though CopyStorm/Restore will always restore records in the optimal dependency order, circular relationships can make the order ambiguous. Consider the following case we have seen with multiple Salesforce instances.

• The Account table has a field called PrimaryContactId — a reference to a Contact record.

• The Contact table has a field called AccountId — a reference to an Account record.

Here is how CopyStorm restores these records.

1. Restore Account records and keep track of all occurrences of PrimaryContactId that are not yet known (because the Contact table has not yet been processed).

2. Restore Contact records.

3. Use the Ids on the restored Contact records to patch the Account.PrimaryContactId fields that were skipped when Accounts were inserted.

It is Step #3 that causes the LastModifiedBy/LastModifiedDate values on Account records to change to the Salesforce credentials running CopyStorm/Restore. Unfortunately there is no other choice since:

• Salesforce does not permit writing to audit fields on update.

• The Account records have to be created before the associated Contact records making step #3 required.

Some Salesforce record types are created as a side effect of creating another record and CopyStorm has no control of over the audit field values for records created by side-effect. Common examples include:

• ContentDocument records are created as a side effect of inserting a ContentVersion record.

• OpportunitySplit records can be created as a side effect of inserting an OpportunityTeamMember.

• ContentFolder records can be created as a side effect of inserting a ContentWorkspace.

There are a number of record types in Salesforce that do not support an INSERT operation when they are in specific state. In this case CopyStorm/Restore will often:

1. Create the record in a state that allows INSERT.

2. Update the record to match the original values in the database record to restore.

Step #2 causes the LastModifiedBy/LastModifiedDate values to be different that those on the original record in the backup. Common examples include Lead and Contract.

It is possible for the side effect of a trigger or workflow to cause an update on a record restored by CopyStorm/Restore. Though CopyStorm/Restore supports disabling of triggers/workflows during a restore, the problem still comes up for several reasons.

1. The CopyStorm/Restore user “forgets” to disable a trigger/workflow.

2. The trigger/workflow is part of a managed package and Salesforce does not support their disablement.

Users can expect 100% accuracy as much as they want but any application using the current Salesforce API will not be able to deliver this level of performance. The only choice we know of for getting 100% accuracy is to hire Salesforce to perform the restore and then pay many $$$ and wait. Once a 100% person sees the cost and wait time we suspect they will live with a few audit fields being different.

When creating data in Salesforce, autonumber columns are given new values automatically. To override this behavior and restore historical autonumber column data with CopyStorm/Restore, there are two approaches that can be taken:

1: Restoring Data to Custom Autonumber Fields

Uploading data to a custom autonumber field requires that autonumber generation be disabled during the restore. It is always best practice to perform this task during a maintenance window to prevent users from creating data while autonumber generation is disabled.

To disable autonumber generation for a custom field while restoring data with CopyStorm/Restore:

1. Go to the custom field in the Object Editor.

2. Record the field information, including the next number in the sequence and the display format.

3. Modify the field type, setting it to Text.

   • This will disable autonumber generation during the restore.

4. Upload the data with CopyStorm/Restore.

5. Modify the field type, changing it back to an autonumber.

   • This will re-enable autonumber generation.

   • Use the field information from Step 2 — potentially with a different sequence number depending on the data being uploaded.

Salesforce’s standard autonumber fields (i.e. Case Number) come with an additional restriction — their field type cannot be modified. Uploading data to a standard autonumber field requires that the standard field be replaced with a copy used in place of the standard field.

To set up a copy of the standard field that can be used with CopyStorm/Restore:

1. Create a new custom field to hold the autonumber data.

2. Create an insert trigger to populate the new custom field:

   • If there is no data in the custom field, populate it by copying the standard autonumber field.

   • If there is data in the custom field, keep the data and don’t overwrite the field content.

After Salesforce is configured with a field to use when restoring the autonumber data, the custom field can be uploaded just like any other field in Salesforce — there are no special configurations needed during the restore process!

The CopyStorm/Restore Record Lineage feature is specifically designed to maintain data record lineage. It keeps recovery information between different incarnations of the same data within the CopyStorm database, ensuring that the relationship between data at different points in its lifecycle is preserved.

CopyStorm/Restore Record Lineage assists with:

• Ensuring data accuracy. Providing the history of data over its lifetime, you can verify its accuracy and identify potential discrepancies.

• Improving auditability and compliance. Maintaining a record of different versions of data allows you to trace changes made to the data through its full lifecycle.

• Streamlining recovery. Keeping track of all versions of data, even when data recoveries may have changed its unique ID many times, means that you can ensure that data recoveries will not introduce duplicate data.

Record Lineage refers to the existence of a logical record across one or more physical forms. For example, an ACME Account can exist in both Salesforce Production 1 and Salesforce Production 2. These are logically the same Account, but are represented by two distinct physical records in separate environments.

Lineage is the mechanism by which CopyStorm/Restore tracks and links these physical records back to their single logical identity, enabling comparison & synchronization across systems.

Lineage plays a role in several key stages of the CopyStorm/Restore application:

• Record Insertion

• Record Matching

• Candidate Scanning

Lineage is created when Lineage Creation is enabled and a record is INSERTED to Salesforce.

Example:

1. Lineage Creation is enabled.

2. ACME Account is inserted into Salesforce

3. CopyStorm/Restore records a new lineage entry for ACME by creating a new row in the CopyForceRecordLineage database table.

The newly inserted ACME Account will have a new Salesforce ID, and is now considered the authoritative (HEAD) revision of the ACME lineage.

There is no limit to the number of records that can exist within a single lineage.

By default, CopyStorm/Restore uses an . However, when considering lineage during restoration, this approach can fall short—because a single logical record may exist in multiple environments with different Salesforce IDs.

To address this, CopyStorm/Restore examines the lineage of the target record to identify the best match in Salesforce.

The general rule of thumb for Lineage-based matching is:

• The closest valid lineage match in Salesforce will be used as the record match.

Note: When using Multi-Column or External ID matching policies, those methods will take priority. Lineage-based matching will only be used as a fallback if no matches are found through the primary matching policy.

The user wants to UPDATE the “Phone” field for HEAD ACME in Salesforce with the “Phone” value from an older ACME record.

1. CopyStorm/Restore identifies a legacy (non-HEAD) lineage ACME Account for restoration.

2. The legacy record has a different Salesforce ID than the current HEAD revision.

CopyStorm/Restore will:

1. Retrieve all Lineage information for the legacy ACME record.

2. Search Salesforce for existing lineage entries, starting from the HEAD.

3. Identify the most recent matching record in Salesforce.

   1. In this case, the HEAD exists in Salesforce, so HEAD will be identified as the match.

There are three major components to the candidate scanning process:

1. Top Level Candidate Scan

   1. Find all records matching the

2. Reference Candidate Scan

   1. Find all parent records that are referenced by the results of the top level candidate scan.

CopyStorm/Restore provides configuration options to enable/disable Lineage consideration during all phases of the candidate scanning process:

• Enable Record Lineage Candidate Scanning

  • Enables Lineage consideration when scanning top-level candidates.

• Related Lists Scan Options

  • Controls how Lineage is considered when scanning child records.

When restoring records with lineage enabled, users must specify Point-In-Time configuration parameters to define which version of the record should be restored.

This configuration is determined by the Point-In-Time Strategy selected in the “Recover” drop-down menu.

• Most Recent: Recover the HEAD of lineage.

• A Point In Time: Recover the version of selected records closest to the selected date / time.

• A Single Date: Recover records that were modified on a given date.

• Between Two Times: Recover records modified between to dates / times.

CopyStorm/Restore will apply record selector filters & consider Point-In-Time Configuration to determine which lineage entries are top-level candidates

When restoring records, users can control how parent or reference records are evaluated as candidates using the following Reference Candidate Scan options:

• Parent Records will not be added as candidates unless they are HEAD or the referenced record has no lineage.

• This option ensures that restored reference relationships remain consistent with HEAD lineage entries in Salesforce.

• Parent records are only added as candidates if they belong to a lineage and the HEAD of that lineage exists in Salesforce.

• In these cases, the reference ID in the top-level record will be replaced with the HEAD ID from the lineage.

• If a reference record has no lineage, the original reference ID is left unchanged.

• Parent records will be added as candidates regardless of whether they are the HEAD of their lineage or whether they establish a consistent relationship. All lineage versions are treated as valid candidates.

When restoring records, users can control how child records (typically related lists) are evaluated as candidates using the following Related Lists Candidate Scan options:

• Only HEAD lineage records are considered valid child candidates.

• Child records will not be added as candidates unless they are the HEAD of their lineage.

• Ensures that restored child relationships remain consistent with HEAD lineage entries in Salesforce. Child records are only added as candidates if they belong to a lineage and the HEAD of that lineage exists in Salesforce.

• In these cases, the relationship will point to the HEAD record.

• If a child record has no lineage, it will be used as-is.

• Child records will be added as candidates regardless of whether they are the HEAD of their lineage or whether they establish a consistent relationship.

• All lineage versions are treated as valid candidates.

Record Lineage data is stored in the CopyStorm Database in a table called CopyForceRecordLineage.

There are a number of reasons that may cause CopyStorm/Restore to skip a record during restore. The most common include:

• The record references another record that is not in the CopyStorm database.

• The record references a record that failed to restore for another reason.

• The record is already in the target Salesforce and has exactly the same timestamp.

There are also a number of Salesforce table specific reasons that will cause a record to be skipped (i.e. operations Salesforce does not allow). For example,

• Activated Contracts cannot be written to Salesforce.

• Converted Leads cannot be written to Salesforce.

• Opportunity Line Items cannot be inserted if they reference an inactive price.

• Quotes cannot be inserted if they reference an inactive price book.

Normally, when a record is skipped the reason is stored as part of the tracker database (an exception to this rule is make for Converted Leads and Activated Contracts). When there is a reason in the tracker database it can be found by clicking on the “Status” of a record — the reason will be displayed in a popup dialog. If no popup dialog appears, CopyStorm/Restore did not record the reason the record was skipped.

There are a few options on the “Global Parameters” tab that can increase the number of skipped record reasons that are recorded. These options can be found under the “Skipped Record Reporting Options”. If you enable all options in this section, the maximum number of “skipped record reasons” will be recorded in the tracker database.

Capstorm is commonly asked “How can I do a complete restore of my Salesforce?”, and the short answer is you cannot. This article explains why and the kinds of restores that occur in practice.

Background

For a person coming from a traditional database world, how to do a complete restore is a simple question. Traditional databases allow an operator to easily disable database access and replace the image with a backup — this technique allows an operator to put a database into a known state at any desired point in time.

Salesforce, however, is unlike a traditional database in several important ways, making a traditional database restore impossible:

• It is impossible to get an exact point in time backup of Salesforce using the Salesforce API.

• Salesforce cannot be shutdown during the restore process. This often means that data entered by employees may be overwritten during while the restore is in progress.

• The restore process will change Salesforce.

  • A restore is analogous to hundreds of data entry clerks rapidly entering data into Salesforce.

  • Salesforce has no idea that the “clerks” are really a restore process, as API calls and “clerks” look the same to Salesforce.

• Not all tables and metadata in Salesforce are writable from the API.

• There is no such thing as an empty Salesforce database — even an “empty” sandbox has dozens (or more) tables populated with data.

There is another Salesforce “twist” that can further complicate restores. Restoring Salesforce metadata (e.g. ApexClasses, Components, PageLayouts, Triggers, SObjects, etc…) often requires dependency knowledge that is not necessary when restoring a traditional relational database. This knowledge is not always provided by the Salesforce API. Since API limits the size of a single metadata deployment, the knowledge of metadata dependencies is an absolute requirement. Total automation of a metadata restore would require a “compiler/semantic analysis” solution that understands the dependencies between all Salesforce metadata types. With the exception of internal Salesforce development, there is no absolute authority on how to perform this dependency analysis.

In practice there are three general classifications of restores used — Sandbox, Production, and Migration.

The main objective of a sandbox restore is to populate an empty development sandbox with a rich subset of data from a production backup. The sandbox generally has one of the following purposes:

• To provide a rich set of data for a developer.

• To provide a rich set of data for a trainer to use in a class.

  • In this case, multiple sandboxes are populated using the same restore process.

Since sandboxes rarely have enough space to hold all of the production data, a user needs to select the data important to their task. For example, a developer may want to restore all Opportunities won in the past 6 months with a value greater than $50,000 along with their associated Accounts, Contacts, Cases, and related custom objects. The process a developer would follow is:

• Select the records and relationships to restore.

• Press a button to restore the records and relationships.

• Optionally add additional records and relationships.

• Press a button to restore the additional records and relationships.

A less frequent type of sandbox restore is to refresh the data in a sandbox from current production data.

CopyStorm/Restore supports all of these restore cases and sees many of them every day.

Production restores are more varied than sandbox restores but tend to fall into these categories:

• Column Restore

  • Restore a handful of columns in a table due to data corruption.

• Hierarchy Restore

  • Restore an Account (or another top-level record) and all associated tables after it was accidentally deleted and removed from the recycle bin.

The common characteristic of production restores is that an operator has surgically precise knowledge about what needs to be restored.

Though it is technically possible for a massive restore to involve hundreds of tables and millions of records on a production Salesforce, this would mean that either the Salesforce infrastructure has failed or your Salesforce data governance has suffered a massive failure/breach. In practice this almost never happens. The worst production disasters Capstorm has seen have involved:

• A Salesforce data center failure lost 6 hours of production updates.

• A key logger corrupted specific fields in a customer’s Salesforce for a period of 2 months.

• A programming error deleted a large set of attachments.

• A mistake in a trigger corrupted a known set of fields.

From a data recovery standpoint, all of these were surgical level restores since there was a level of precision available to identify the corrupted or missing data.

Migration restores involve moving large hierarchies of records from one Salesforce instance to another where:

• The target Salesforce instance already contains a lot of records.

• The source instance records may or may not match records already in the target.

  • How to determine record matching varies by record type.

• The table structure of the two instances is not the same.

Doing a migration is generally much more complex than a sandbox or production restore. The core issue is a classic ETL problem:

• Extract the data from the source Salesforce.

• Transform the data into the structure required by the target Salesforce.

• Load the data into the target Salesforce.

Capstorm products deal with the Extract problem (CopyStorm) and the Load problem (CopyStorm/Restore). The Transform problem is highly specific to each migration and is what typically makes a migration project difficult.

Another type of migration restore is used by several Capstorm customers — bringing data from many disparate systems into Salesforce. In these cases:

• Data from many sources is populated into a CopyStorm created schema using customer written code.

• CopyStorm/Restore is used to restore the data into a production Salesforce.

The highest volume example of this type of migration task that Capstorm knows about restores millions of records every day collected from over 50 disparate data sources.

If your requirement is to “click a few boxes and restore any Salesforce instance to the exact state of a backup” then we wish you good luck. However, if your operational requirements fall into the restore categories described above the Capstorm suite of products is your best choice. Of course, you could write the tools in-house and discover the less documented underbelly and restrictions of the Salesforce API on your own time. If you want to be convinced that there is an underbelly, check out this article we posted a few years ago: