Salesforce records often contain references to other records. For example:
Contact records contain a reference to an Account in their AccountId field.
Nearly all records contain a reference to the Salesforce user who owns the record in an OwnerId field.
The Reference Fields portion of the Restore Set Editor defines the action which should be taken if a record references another record that is neither:
Already in Salesforce.
Restored by another table rule.
This editor contains three columns for each reference field:
A checkbox which determines whether or not the reference field should be restored.
If not checked the field will not be modified on an UPDATE and will take the default value on an INSERT.
The name of the reference field with a superscript indicating the Salesforce table to which it refers.
The action to take if the value of the reference field in CopyStorm points to a record not present in the target Salesforce.
The following screen shot lists the reference fields for the Account table in a default Salesforce instance:
For all reference fields EXCEPT those referencing the Salesforce User table, the following options will be available:
Reference field linking to the Salesforce User table have a different set of options. Common reference fields of this type include “OwnerId”, “CreatedById”, and “LastModifiedById”.
Salesforce User references are treated differently because Salesforce will not permit a record to be created with an owner that is an inactive Salesforce User.
Set to Null
If the referenced record is not already in Salesforce then set the reference field to null.
Skip Record
If the referenced record is not already in Salesforce then do not restore the record.
Create Record
Use Default
If the referenced User record is missing or is an inactive Salesforce User then use the Salesforce User specified as the default User on the “Global Parameters” tab.
Skip Record
If the referenced User record is missing or is an inactive Salesforce User then do not restore the record.
If necessary, create the referenced record in Salesforce.
Configure Rich Text Fields for pre-processing before Sending to Salesforce. Options:
Patch RT Images
Replaces all Servlet/RTA Image references with their Base64 encoded image data.
Note: Upon record insert/update, Salesforce will re-convert the Base64 body to an RTA Image.
Patch RT URIs
Finds and updates Salesforce record IDs embedded in RTA fields. When a record is restored or reinserted with a new ID, this option rewrites hardcoded references to reflect the correct restored record IDs
Supported Formats:
/001XXXXXXXXXXXXXXX
NOTE: Only IDs that can be mapped using the Restore’s ID Map or Tracker Database will be replaced. Others remain unchanged. For more information on these functions, see , and .
Salesforce records can hold many different formats of images in URLs. Making use of this configuration file allows for the addition of different Regex strings to allow support for new image URL formats in the field.
Setup
Navigate to “CopyStormRestore/Config” directory
Create an xml file and save it as “SFDCRecordIdTextReplaceUtilConfig.xml” inside of the “CopyStormRestore/config” directory.
The xml file consists of two tags:
The config file opening/closing tag.
See below for the default configuration file:
The “Related Lists” editor determines which records from related lists will be restored for each record. For example, when restoring Account the related lists Contacts, Cases, and Opportunities will usually be available.
To restore a related list click on the check box next to the name.
In the following screen shot the related lists Assets, Cases, Contacts, and Opportunities were selected for the Account table. This will cause CopyStorm/Restore to restore all Asset, Case, Contact, and Opportunity records related to selected Account records. Note that the superscript on each related list indicates the SalesForce Table and Column associated with the list.
Related lists can be selected to any level. Example: Account->Contact->Attachments.
As related lists are selected, they will automatically be marked as checked in the master table list on the left.
The control buttons at the top are often quite important and can be used to:
Expand/Collapse the tree of related lists.
Select all related lists (direct, grandchildren, great-grandchildren, .etc.) to any level.
Deselect all related lists.
In the following screen shot, the account named Capstorm, LLC and all of its related lists were selected for a restore. The steps were:
Set the Account filter to select just the Capstorm, LLC company.
Click on the Select All button for related lists.
Are you curious about how many tables were actually selected? The answer is 60!
When restoring test datasets to developer sandboxes or other environments with limited data capacity, it can be useful to limit the total number of records that will be restored either in:
Any single related list under any single record – for example, to ensure that no single Account will have more than the specified number of Contacts.
A single table, regardless of how many related records are in the CopyStorm database, will be limited to the specified number of records.
In practice, it is unusual to restore absolutely every related list for a record because certain system tables are best excluded (since they are already populated). When the Select All button is clicked, checking the “Include System Tables” checkbox will give you the option of restore none, a few, or all related system tables.
%2Fr%2FObject__c%2F001XXXXXXXXXXXXXXX%2Fview
Regex Entry tags. Each Regex entry tag has two components, a unique name or “type” that is a unique identifier for the entry, as well as a Regex string used to match against URL patterns. The capture group of these Regex strings needs to be the Salesforce ID of the image resource for pre-processing to be successful.
<SFDCRecordImageUtilConfig>
<!--
The different types of SFDC image URL formats and their associated REGEXs to
strip out the target image information.
SFDC records can hold many different formats of images in URLs. This configuration
file allows for the addition of different REGEX strings to allow support for new
image URL formats in the field.
NOTE: The "<" and ">" characters are reserved in XML and cannot be a part of the "regex"
parameter. Replace those characters or other XML reserved characters with the following:
- "<" -> "<"
- ">" -> ">"
- (") -> """
-->
<!--
-> REGEX String:
-> (?:/r/[a-zA-Z0-9_]+/)([a-zA-Z0-9]{18}|[a-zA-Z0-9]{15})(?=/|$)
-> REGEX String Details:
-> Pattern: (?:/r/[a-zA-Z0-9_]+/)([a-zA-Z0-9]{18})(?=/|$)
-> (?: ... ): Non-capturing group, meaning it groups the enclosed pattern but does not store the matched value.
-> /r/: Matches the literal string /r/.
-> [a-zA-Z0-9_]+: Matches one or more (+) alphanumeric characterse and the underscore. This is used to match
to the Salesforce object name in the URL.
-> /: Matches the literal forward slash (/) after the Salesforce object name in the URL.
-> ([a-zA-Z0-9]{18}|[a-zA-Z0-9]{15}): Capture group, meaning this is the matched content that is stored. Matches exactly 15 or 18 alphanumeric
characters for the Salesforce record ID.
-> (?=/|$): Ensures that the match is only valid if the capture group is following by a '/', or it is the end of the string
Parses URLs in the formats:
-> https://exampleorg.lightning.force.com/lightning/r/Knowledge__kav/001Ec00000L3Pz4IAF/view
-> https://exampleorg.lightning.force.com/lightning/r/ContentDocument/0696Q00000CTZ9mQAH/view
-->
<SFDCImageFormat type="urlIdNoReference" regex="(?:/r/[a-zA-Z0-9_]+/)([a-zA-Z0-9]{18}|[a-zA-Z0-9]{15})(?=/|$)"></SFDCImageFormat>
<!--
-> REGEX String:
-> (?:%2Fr%2F[a-zA-Z0-9_]+%2F)([a-zA-Z0-9]{18}|[a-zA-Z0-9]{15})(?=%2F|$)
-> Pattern: (?:%2Fr%2F[a-zA-Z0-9_]+%2F)([a-zA-Z0-9]{18})(?=/|$)
-> (?: ... ): Non-capturing group, meaning it groups the enclosed pattern but does not store the matched value.
-> %2Fr%2F: Matches the URL encoding for the literal String (/r/)
-> [a-zA-Z0-9_]+: Matches one or more (+) alphanumeric characterse and the underscore. This is used to match
to the Salesforce object name in the URL.
-> %2F: Matches the URL encoding for the literal string forward slash (/).
-> ([a-zA-Z0-9]{18}|[a-zA-Z0-9]{15}): Capture group, meaning this is the matched content that is stored. Matches exactly 15 or 18 alphanumeric
characters for the Salesforce record ID.
-> (?=%2F|$): Ensures that the match is only valid if the capture group is following by a '/', or it is the end of the string
-> Parses URLs in the formats (Gets the second ID in the URL):
-> https://exampleorg.lightning.force.com/lightning/r/Knowledge__kav/001Ec00000L3Q00IAF/view?ws=%2Flightning%2Fr%2FKnowledge__kav%2F001Ec00000L3Q01IAF%2Fview
-->
<SFDCImageFormat type="urlIdSingleReference" regex="(?:%2Fr%2F[a-zA-Z0-9_]+%2F)([a-zA-Z0-9]{18}|[a-zA-Z0-9]{15})(?=%2F|$)"></SFDCImageFormat>
<!--
-> REGEX String:
-> (?:sfdc://)([a-zA-Z0-9]{18}|[a-zA-Z0-9]{15})
-> REGEX String Details:
-> (?:sfdc://)
-> Non-capturing group to find "sfdc://" right before the capture group
-> ([a-zA-Z0-9]{18}|[a-zA-Z0-9]{15})
-> Capture group matching alphanumerics 18 characters in length and if not, 15 characters in length.
-> Parses rich text in the format:
-> <p>this is test html<img src="sfdc://001ZZrecord7890123" alt="first tag"/>with two image tags<img src="sfdc://001ZZrecord7890123" alt="second tag"></img></p>
-->
<SFDCImageFormat type="sfdcTag" regex="(?:sfdc://)([a-zA-Z0-9]{18}|[a-zA-Z0-9]{15})"></SFDCImageFormat>
<!--
-> REGEX String: https://[^/"]*/([a-zA-Z0-9]{15,18})(?=(?:/)?")
-> REGEX String Details:
-> https://
-> Matches the HTTPS protocol
-> [^/"]*
-> Non-capture group to match everything after the domain name but stop at the next slash or quote, effectively skipping the host portion
-> /([a-zA-Z0-9]{18}|[a-zA-Z0-9]{15})
-> Captures the Salesforce ID after a slash (/) in the 18 or 15 character format
-> (?=(?:/)?")
-> Positive look ahead to make sure that after the pattern for the ID is either a quote (") or a slash and quote (/") for termination of the match
-> Parses rich text in the format:
-> <p><a href="https://example-data-12345-dev-ed.scratch.my.salesforce.com/kA0Ov000000S6IbKAK" target="_blank">Parent Article</a></p>
-->
<SFDCImageFormat type="urlIdNoObjectName" regex="https://[^/"]*/([a-zA-Z0-9]{18}|[a-zA-Z0-9]{15})(?=(?:/)?")"/>
</SFDCRecordImageUtilConfig>
The “Fields to Restore” editor determines which fields for each record will be restored to Salesforce.
By default all fields are selected.
If a field is not selected it will:
Be set to null for records inserted into Salesforce.
Be left unchanged for updated records.
Fields required for creating a new Salesforce record are marked as (required).
If any required fields are not checked then only update restore operations are permitted.
The editor contains filtering tools above a list of all available table fields. In the following screenshot the filter has been used to only show fields whose name contains the text “Billing”:
Other “Fields to Restore” options:
The “Field Obfuscation & Masking” button launches a dialog used to control how individual fields are restored.
The “Transformer” column indicates that a data transformation should be applied BEFORE data is written to Salesforce. In the example screenshot two columns have been assigned to transformers. Available transformers include:
Asterisk
Converts a column to all **** before writing the field to Salesforce.
For example, “Rebecca” will become “*******”.
Email Scrambler
If the built-in Transformers do not perform the action you need, CopyStorm/Restore supports adding new Transformers. Contact if you want to learn how this is done.
Other columns on this dialog provide read-only information about each column in the corresponding table.
The Restore Set Editor tab is used to select which tables and fields will be restored to Salesforce. The editor is divided into two major sections:
The Table Selector
The Table Editor
The section on the left, the Table Selector, indicates which table(s) have been selected for the restore.
Tables that have been selected to be restored will always be displayed.
Randomizes email address fields before writing them to Salesforce.
For example, the email “luck@casino.com” may become “AuFK@uuiLT.com”.
Email Append Invalid
Appends “.invalid” to email address fields before writing them to Salesforce.
For example, the email “luck@casino.com” will become “luck@casino.com.invalid”
Lorem Ipsum
Change a block of text to the infamous “Lorem ipsum dolor sit amet, consectetur adipiscing…”
Random: Account Name / City Name / Country Name / Street Name / First Name / Last Name / Job Title
Converts a column to a value selected randomly from a dictionary.
The value will be selected from a built-in dictionary containing values for the indicated type (e.g. “Random: Account Name” will use a dictionary of Account Names).
Uniform: Account Name / City Name / Country Name / Street Name / First Name / Last Name / Job Title
Converts a column to a value selected non-randomly from a dictionary. The same input value will always provide the same output value.
The value will be selected from a built-in dictionary containing values for the indicated type (e.g. “Uniform: Account Name” will use a dictionary of Account Names).
Scramble
Converts each letter or digit to a random character of the same class before writing the field to Salesforce.
Supports all numerical fields including phone, integer, or double.
For example,
“Rebecca” may become “Kiojjkl”
“567-45-0987” may become “778-33-0997”
“+44 45 345 8876” may become “+62 33 123 9567”
Same Year Date/Time
Converts a date or date/time field to a random date or date/time within the same year.
Selected
If checked, only selected fields will be displayed.
Invert
When pressed, the field selection will be inverted. All selected fields will be deselected, and all non-selected fields will be selected.
Clear
When pressed, all selected fields will be deselected.
Tables that have not been selected will appear only if their name matches the filter criteria under the “Select” drop-down located in the top of the selector.
If a table name is followed by a red (ref), only records referenced by other tables in the list will be restored.
If a table is selected but no longer in Salesforce its name will be followed by a red [Not in Salesforce].
The section on the right, the Table Editor, is used to control the restore parameters for the selected table on the left. In the following screen shot a table has not yet been selected. Click on a table name to display its Restore Rule.
The Table Editor on the right-hand side of the Restore Set Editor tab controls restore parameters for individual tables. It is divided into 4 or 5 tabs.
The “Select Records” tab controls:
Which specific records will be restored to Salesforce.
Whether Inserting, Updating, or both operations should be allowed.
The “Reference Fields” tab controls what should happen when a table’s reference fields are pointing to records not in the target Salesforce and not part of the restore. For example, when restoring an Account with a Parent Account should the Parent Account be automatically restored.
Available options include “Restore Referenced Record” and “Skip Record”.
The “Fields” tab supports fine control over which fields will be restored AND field level masking and obfuscation. By default all fields are restored.
The “Related Lists” tab supports including records in child tables, grandchildren, great-grand children, etc. (to any depth) in the set of records to restore. Example: With a couple of mouse clicks CopyStorm/Restore can be told to restore all child, grandchild, etc. tables for selected Accounts.
Record Filters in CopyStorm/Restore provide an easy way to limit record selection without writing SQL. Following these steps will enable CopyStorm/Restore to restore a specific set of records to Salesforce.
To limit restored records to specific criteria:
Select the “Filter” in the “Which Records” dropdown.
Click on the empty filter to edit.
Create the criteria by dragging and dropping fields and references.
Test the filter and validate the criteria.
Press “Save” to apply the filter.
To enable filter-based record selection for a table:
Select the table in the Restore Set Editor.
Update the “Which Records” selector to “Filter”.
Click on the “Click Here To Edit” button to open the filter editor.
Edit the filter by dragging and dropping fields and operators from the left-hand side of the dialog into the right-hand tree.
Referenced data can be filtered by expanding reference fields in the field tree.
Click on an item in the right-hand filter tree to edit the criteria. Pressing “Save” will save changes in the criteria editor, and pressing “X” will delete the criteria from the filter.
To test the filter and verify that it is finding the correct records, press the Preview button. This will query the CopyStorm database for a sample set of records that match the criteria and display them in the table.
To view all fields in the query results, select “All Fields” in the “Show” drop-down. To view the SQL query that was generated, press the “Show Query” button.
If the query results are correct, press “Save” to save changes to the filter and close the dialog. To close the dialog without saving changes, press the “Cancel” button.
Parameters in this tab are less likely to be configured by the average user. They include:
Max # Per Update
Maximum number of records to be sent to Salesforce in a single batch. By default use the parameter from the Global Parameters tab. Never set this value unless something like an ApexTrigger or Workflow is limiting the number of records that can be written in a batch. This condition typically occurs as a result of poorly written Apex code.
A handful of Salesforce tables can be children of many other types of tables. Good examples include Attachments and Notes.
If you want to include the Attachments for all selected tables, one way is to open the restore rule for each table and include Attachments as a related list. Another quicker way is to open the Attachments editor and quickly select all possible parents using a GUI extension that only appears on tables like Attachments.
On the screen shot, Attachments for Contacts, Leads, etc. can be included in the restore set simply by checking the box next to the parent table name. Though this action is equivalent to adding Attachments as a related list, you will find this technique must faster for larger restore sets.
# Salesforce Writers
Maximum number of concurrent write request to Salesforce. It is rare to set this value because CopyStorm/Restore dynamically selects a value which gives optimal performance as part of its runtime Salesforce analysis.
Skip Record Id Check
When enabled, CopyStorm/Restore will avoid the check to see if a record is already in the target Salesforce. Note that when “Skip Record Id Check” and “Skip Original Id Check” are both enabled, all operations will be Inserts.
Skip Original Id Check
When enabled, CopyStorm/Restore will avoid the check to see if a record has already been restored to the target Salesforce by checking the “Original Id” column. Note that when “Skip Original Id Check” and “Skip Record Id Check” are both enabled, all operations will be Inserts. Note that this option will only be displayed if the “Original Record Id” Migration feature is enabled.
The “Select Records” tab on the “Restore Set Editor” determines which records will be restored to Salesforce. The “Select Records” tab is divided into two major sections:
The top section is common to all techniques available for finding records to restore.
The bottom section contains parameters unique to a specific record finding approach.
The following image illustrates the primary elements in this tab.
The “Which Records” option provides a large number of techniques for determining what records to restore. The techniques include:
The “Restore Type” options determines what kinds of operations are allowed when writing to Salesforce.
Updates only
Inserts only
Both Inserts and Updates
By default both inserts and updates are allowed unless a required field is excluded. When a required field is excluded then Inserts are automatically disallowed.
The “Record Match Policy” options determines how CopyStorm/Restore decides whether to update an existing record or and insert a new record when writing to Salesforce. The follow policies are supported:
(Note: for all polices a CopyStorm->Salesforce Id match check will be attempted if the base policy fails to find a match).
The following two images illustrate the “External Id” and “Multi-Column” record matching policies
Other parameters in the top section are less common to be set by a user. They include:
The “All” option will restore all records in the CopyStorm backup to the Salesforce instance:
The “Filter” option will restore records that match a drag-and-drop set of criteria in the CopyStorm backup to the Salesforce database. To use a record filter to select records in CopyStorm, please see the page here: .
The “Id File” option will read “ids” from a file and restore all records which match the supplied ids.
The field to treat as the “id” is selected from the pull down menu. Of course, a 15 or 18 character Salesforce Id will always work (and is the default).
However, all Salesforce Id and text fields can be used. It is OK for the Id to match multiple records. For example you could:
Use BillingState as the “id” and select all accounts in Florida and Alabama (the Ids values would be Florida and Alabama).
The “Id List” option will restore Salesforce records based on record Ids copied into a text box. Ids should be delimited using a space or a comma.
There is no practical limit to the number of Ids that can be pasted into the box.
Any Salesforce Id type column or text column may be chosen to use as the “id” field.
Note: For real Salesforce Ids use a 15 or 18 character id.
The “Random” option will restore a random number of Salesforce records, optionally limited to a set of Ids provided based on a SQL statement or record filter.
If a value is provided for the “Seed” parameter the same records will be restored each time.
This can be useful for restoring a development sandbox to exactly the same state every time.
The “Seed” parameter can be any positive non-zero number from 1 to 9 digits in length.
To restrict records using a record filter, select “Filter” from the “Constraint Type” dropdown and:
Click on “Click Here To Edit”.
See this page for more information on configuring a record filter in CopyStorm/Restore: .
To restrict records using a SQL statement, select “SQL” from the “Constraint Type” dropdown and:
Example 1: Limit the selection to accounts with a shipping country of “Elbonia”:
SELECT id FROM Account WHERE ShippingCountry='Elbonia'
Example 2: Limit accounts to those with Opportunities having a value greater than $1,000,000:
The “Referenced” option will only restore a record if it is referenced by another Salesforce record being restored. This option is selected automatically as a side-effect of including a related list for another table.
For example, if a user selects random Account records to restore and selects the Opportunity related list, a rule to restore referenced Opportunities will automatically be created.
When creating test data in a sandbox restoring referenced records may cause more data than desired to be restored. For example, if restoring all Cases referenced by restored Accounts, much more data than needed may be restored.
There are three parameters which can control the number of referenced records to be restored:
The “Max # Per Parent” is the maximum number of child records that should be restored for each parent.
For example, if this number is 5 for restoring Cases, then no more than 5 Cases will be restored for each unique Account.
The “Max # Total” is the maximum number of child records that should be restored for all parents.
Note that the actual number of referenced records restored may be slightly larger than the specified constraint. This is because records are also restored because of a reference field constraint, but the number of additional records restored is typically small.
The “SQL” option will restore all records whose id matches the value returned by a user supplied SQL statement.
For example,
To select Contacts in Ohio:
SELECT id FROM Contact WHERE MailingState='OH'
To select Contacts with open support cases:
The “SOQL” option will restore all records whose id matches the value returned by a user supplied SOQL statement.
For example,
To select a Contact record by name:
SELECT id FROM Contact WHERE Name='Michael Jordan'
To select Contacts with closed support cases:
Paste in a list of Ids listing the records to restore. The “ids” can be any Salesforce “Id” or “String” type column.
Restore a random number of records, optionally using a filter to control which records can be selected.
Only restore records which are referenced by another table.
Write SQL against the CopyStorm database to determine which records to restore. This technique is for advanced developer types.
Write SOQL against the target Salesforce instance to determine which records to restore. This technique is for advanced developer types.
None
No records will be restored. This can be useful to suppress restores to a table when pending records are already in the tracker database.
Skip Original Id Check
When enabled, CopyStorm/Restore will avoid the check to see if a record has already been restored to the target Salesforce by checking the “Original Id” column. Note that when “Skip Original Id Check” and “Skip Record Id Check” are both enabled, all operations will be Inserts. Note that this option will only be displayed if the “Original Record Id” Migration feature is enabled.
SELECT DISTINCT accountId FROM Opportunity WHERE amount > 1000000The “Constraint Type” allows the restored records to be limited to referenced records that match a specific set of criteria.
Select “Filter” to edit a drag-and-drop record filter that will be used to limit the related objects being restored.
For documentation on using a record filter with CopyStorm/Restore, see the page: Limiting Records To Restore With a Record Filter.
Select “SQL” to provide a SQL fragment that will be added to the WHERE cause of the SQL used to look for related objects.
For example, if this is set to “closeDate >= ‘2015-01-01′” for Opportunity then only Opportunities closed in 2015 or later will be selected.
Any column in the CopyStorm backup database can be used to form this constraint.
The only restriction for this parameter is that it must be valid SQL when appended to a statement like:
SELECT id FROM Opportunity WHERE (...) AND (<>)
SELECT contactId FROM Case2 WHERE isClosed=0SELECT contactId FROM Case2 WHERE isClosed=1Restore all records from the CopyStorm backup.
Use a rich GUI to create a filter limiting which records are restored.
Provide a file of Ids listing the records to restore. The “ids” can be any Salesforce “Id” or “String” type column.
Salesforce Id
If the record Id in CopyStorm matches an existing Id in the target Salesforce then an update operation is performed. Otherwise an insert operations is performed.
External Id
If a specific External Id column in CopyStorm matches the same column in Salesforce then an update operation is performed. Otherwise an insert operations is performed.
Multi-Column
Max # Per Update
Maximum number of records to be sent to Salesforce in a single batch. By default use the parameter from the Global Parameters tab. Never set this value unless something like an ApexTrigger or Workflow is limiting the number of records that can be written in a batch. This condition typically occurs as a result of poorly written Apex code.
# Salesforce Writers
Maximum number of concurrent write request to Salesforce. It is rare to set this value because CopyStorm/Restore dynamically selects a value which gives optimal performance as part of its runtime Salesforce analysis.
Skip Record Id Check
If one or more specified columns matches the same columns in Salesforce then an update operation is performed. Otherwise an insert operations is performed.
When enabled, CopyStorm/Restore will avoid the check to see if a record is already in the target Salesforce. Note that when “Skip Record Id Check” and “Skip Original Id Check” are both enabled, all operations will be Inserts.