Does the CopyStorm Database Have Logging Tables?
Each time CopyStorm runs it records a variety of data into tables in the CopyStorm database:
- A summary of records written to the database and any exceptions.
- The content and structure of the tables will be familiar because they mimic what is displayed as progress within the CopyStorm GUI.
- Changes to field values (see CopyForceFieldHistory in this article).
- DDL statements (e.g. ALTER TABLE) that were issued against the CopyStorm database.
CopyStorm database logging tables were introduced starting with the 7.*.* version of CopyStorm.
The following tables are used for logging:
| Table Name | Description |
|---|---|
| CopyForceJob | Contains a single record for each time CopyStorm runs. |
| CopyForceJobTable | Contains a single record for each table modified in a CopyStorm run. |
| CopyForceJobException | Contains a record of each exception (problem) encountered during a CopyStorm run. |
| CopyForceDDLLog | Contains a record of all DDL type statements issued against the database. |
| CopyForceFieldHistory | If field history tracking is enabled, this table will record the old and new value each time a field changes (except on record insert). |
| CopyForceMedicJob | Contains a single record for every time a CopyStorm/Medic job runs. |
| CopyForceMedicJobException | Contains a record of each exception (problem) encountered during a CopyStorm/Medic run. |
These logging tables are created automatically by CopyStorm. The only changes that should ever be performed by a customer is the deletion of log records — this should be done using CopyStorm/Medic.
The simplest way to view and manage CopyStorm log data is often the Log Manager tool in CopyStorm/Medic.
The remainder of this article describes the column structure for each table.
CopyForceJob
The CopyForceJob table contains one record for each time CopyStorm successfully connects to Salesforce, connects to the target database, and starts a copy.
The table contains these columns:
| Column Name | Data Type | Description |
|---|---|---|
| jobId | integer | Unique ID for this job. This value is the job’s primary key. |
| startTime | timestamp | The time when the job started. Note that this time will be slightly after the start of the CopyStorm executable because it takes a few seconds for CopyStorm to initialize and connect to its data sources. |
| endTime | timestamp | The time when the job completed. This time is updated as each table finishes and after the entire job completes. |
| nCopied | integer | The total number of records copied to the target database. |
| nImagesCopied | integer | The total number of image records copied to the target database. |
| nImagesRead | integer | The total number of image records read from the target Salesforce Org. |
| nDeleted | integer | The total number of records deleted from the target database in accordance of the configured retention policies. |
| nFieldHistory | integer | The total number of field history records inserted into the target database. |
| nArchived | integer | The total number of record snapshots inserted into the target database via the Snapshot feature. |
| nTransactions | integer | The number of Salesforce transactions used during the run. |
| nExceptions | integer | The number of exceptions encountered during the run. If this value is non-zero then details for each exception will be recorded in the CopyStormJobException table. |
| configFileName | string | The base name of the CopyStorm configuration file used for the run. This value will be NULL if there was no active configuration file. |
| configFileDir | string | The directory containing the CopyStorm configuration file used for the run. |
| copyStormVersion | string | The version of CopyStorm that created the log record (e.g. 7.34.1). |
| copyStormDir | string | The directory in which CopyStorm was installed when the job ran. |
| elapsedTime | float | The total number of seconds taken by the job to run. |
| processId | integer | The system process id for the CopyStorm process. |
CopyForceJobTable
The CopyForceJobTable table contains one record for each table that was modified during a CopyStorm run.
The table contains these columns:
| Column Name | Data Type | Description |
|---|---|---|
| jobId | integer | The unique ID of the job that modified the table. |
| tableName | string | The Salesforce name of the modified table. |
| startTime | timestamp | The time when CopyStorm started processing the table. |
| endTime | timestamp | The time when CopyStorm finished processing the table. |
| startTimestamp | timestamp | The timestamp of the first record copied from Salesforce when CopyStorm started processing the table. |
| endTimestamp | timestamp | The timestamp of the last record read from Salesforce when CopyStorm finished processing the table. |
| nRead | integer | The total number of records read from Salesforce. |
| nCopied | integer | The number of records inserted or updated in the CopyStorm database. |
| nSkipped | integer | The number of records read from Salesforce and skipped because they were already up to date in the CopyStorm database. |
| nDeleted | integer | The number of records deleted from the CopyStorm database. |
| nImagesCopied | integer | The total number of image records copied to the target database table. |
| nImagesRead | integer | The total number of image records read from the target Salesforce Org. |
| nFieldHistory | integer | The total number of field history records inserted into the target database for this table. |
| nArchived | integer | The total number of record snapshots inserted into the target database for this table via the Snapshot feature. |
| nPerMinute | integer | The average number of records processed per minute. |
| elapsedTime | float | The total number of seconds used to process the table. |
| startSize | integer | The size of the table in bytes when CopyStorm started processing the table. |
| endSize | integer | The size of the table in bytes when CopyStorm finished processing the table. |
CopyForceJobException
The CopyForceJobException table contains one record for each exception that occurred during a CopyStorm run.
The table contains these columns:
| Column Name | Data Type | Description |
|---|---|---|
| jobId | integer | The unique ID of the job where the exception occurred. |
| exceptionId | string | Order in which the exception occurred (starting at 0). |
| tableName | string | The name of the table being processed when the exception occurred. |
| message | string | A short message describing the exception. |
| traceback | string | The full traceback from the exception. |
| exceptionTime | timestamp | The time when the exception occurred. |
CopyForceDDLLog
The CopyForceDDLLog table contains a record of each DDL statement ran against the CopyStorm database.
The table contains these columns:
| Column Name | Data Type | Description |
|---|---|---|
| tableName | string | The name of the table addressed by the DDL statement. |
| command | string | The full DDL command. |
| traceback | string | If the DDL command failed, this field contains the full traceback from CopyStorm. |
| applyDate | timestamp | The date on which the DDL was executed. |
CopyForceFieldHistory
The CopyForceFieldHistory table records changes to field value. This table will be empty unless field history tracking is explicitly enabled.
History records will be maintained even if the referenced table / field is modified or deleted in Salesforce.
The table contains these columns:
| Column Name | Data Type | Description |
|---|---|---|
| id | string | The unique Salesforce record ID of the record that was the source of the data. |
| tableName | string | The Salesforce API table name. |
| fieldName | string | The Salesforce API field name. |
| oldValue | string | The value of the field before it was updated. |
| newValue | string | The updated value of the field. |
| lastModifiedById | string | The unique Salesforce ID of the User that last modified the record. |
| lastModifiedDate | timestamp | The date on which the record was last modified. |
| createdDate | timestamp | The date on which the history record was created. |
CopyForceMedicJob Table
The CopyForceMedicJob table contains one record for each time a CopyStorm/Medic job runs.
The table contains these columns:
| Column Name | Data Type | Description |
|---|---|---|
| jobId | integer | The unique ID of the job. |
| startTime | timestamp | The time when the CopyStorm/Medic job started. |
| endTime | timestamp | The time when the CopyStorm/Medic job finished. |
| elapsedTime | float | The total number of seconds that the job ran. |
| nTransactions | integer | The number of Salesforce transactions used by the job. |
| nExceptions | integer | The number of Exceptions (problems) reported by the job. |
| configFileName | string | The name of the configuration file used to start the job. |
| configFileDir | string | The path to the configuration file used to start the job. |
| medicVersion | string | The CopyStorm/Medic application version. |
| processId | integer | The system process id for the CopyStorm/Medic process. |
| toolName | string | The medic tool ran by the job. This corresponds to the value provided on the -tool command line switch. |
| status | string | The job status – one of Pending, Running, Cancelled, Finished, or Error. |
| output | CLOB | The job’s output. The format of this field will depend on the type of job that was ran. |
CopyForceMedicJobException
The CopyForceMedicJobException table contains one record for each exception that occurred during a CopyStorm/Medic run.
The table contains these columns:
| Column Name | Data Type | Description |
|---|---|---|
| jobId | integer | The unique ID of the job where the exception occurred. |
| exceptionId | integer | Order in which the exception occurred (starting at 0). |
| message | string | A short message describing the exception. |
| traceback | string | The full traceback from the exception. |
| exceptionTime | timestamp | The time when the exception occurred. |