Help! Partition Creation Failed

When partitioning tables containing large volumes of data or Base64-encoded content, the partition creation process can fail mid-way due to various issues, such as:

• Application freezes or crashes

• Transient network errors

• Resource exhaustion (e.g. memory or CPU spikes)

• Timeouts or I/O bottlenecks

These interruptions can leave partitions in an incomplete or inconsistent state.

The Partition Creation process is designed to be incremental:

• Records are copied from Source to Target Partition in an ID-based ordering (lexicographical).

• After each batch of records is copied from Source to Target, the last read Salesforce ID is persisted in the Database.

  • See the CopyForceTablePartitionLoadTask database table.

Note: CopyStorm will refuse to replicate data to a table if it has a Partition Creation In-Progress

If partition creation fails, you can resume the process by navigating back to the Partition Editor Tool. You’ll see the following dialog appear if a partition creation is in-progress:

From this point, you’ll have the option to:

• Resume the Partition Creation process

  • This will start copying data from Source to Target from the Last Read Salesforce ID.

• Cancel the Partition Creation process

  • Note: Cancelling will not remove data already copied if the Source is S3 or a Folder

  • Cancelling a JDBC Partition Creation will delete the new JDBC Partition.

  • Source data will never be deleted during the cancellation process.