How To Capture CopyStorm Runtime Logs

This article provides information on how to capture CopyStorm job execution log data that is output via Standard Error/Out in a text file. This approach is useful for doing advanced troubleshooting in order to capture error symptoms that may not appear within the application GUI or within the logged exceptions of the job history entry found on the “History” tab.

The general approach is as follows:

Open a headless CopyStorm script for editing.
1. This is the script that is scheduled using CRON or Windows Task Scheduler, and is created using the “Batch File” tab of CopyStorm.
Create a directory that will be used to save the application logs.
Edit the batch script to create a log file with the appropriate name in the established directory and write all job execution output to the file.

Windows

Here is an unedited example of a Windows batch script generated by CopyStorm. This script will be modified to output CopyStorm job logs to a file:

Copy to Clipboard

The following code is added after the line that begins with “SET CopyStormConfigFile”

```
SET LOGDIR=C:\Capstorm\CopyStormLogs
```
- This code sets a local script variable that establishes the path to the directory where the log files will be stored.
- The “C:\Capstorm\CopyStormLogs” path should point to your desired CopyStorm log directory path.
```
SET TIMESTAMP=%date:~10,4%-%date:~7,2%-%date:~4,2%T%TIME::=-%
```
- This code sets a local script variable to the current timestamp at which the script is executed. Timestamps will be generated using YYYY-MM-DDTHH-mm-SS.SS formatting – e.g. “2023-02-21T19-45-32.12”.
- You do not need to change any of this code, just copy and paste it directly.

Next, the second to last line (starting with “CALL %CopyStormCmd%”) will need to be modified to write to the logfile. The final line will look like this:

CALL %CopyStormCmd% >> %LOGDIR%\CopyStorm_Log_%TIMESTAMP%.txt 2>&1

This code creates a logfile and redirects stderr and stdout to the file, capturing all job execution logs.

The “CopyStorm_Log_” prefix is used to identify the type of logfile – if you have multiple scheduled jobs, it is best practice to use a different prefix for each scheduled job. For example, “CopyStorm_Hourly_Data_” and “CopyStorm_Daily_Metadata_” if there is an hourly job for data and a separate daily job for metadata.

The example above will create log files with names like:

CopyStorm_Log_2023-02-21T19-45-32.txt

Here is an edited version of the batch script shown above:

Copy to Clipboard

Linux

Here is an unedited example of a Linux sh script generated by CopyStorm. This script will be modified to output CopyStorm job logs to a file:

Copy to Clipboard

The following code is added after the line that begins with “CopyStormConfigFile=”

```
LOGDIR="/opt/Capstorm/CopyStormLogs"
```
- This code sets a local script variable that establishes the path to the directory where the log files will be stored.
- The “/opt/Capstorm/CopyStormLogs” path should point to your desired CopyStorm log directory path.
```
TIMESTAMP=$(date +"%Y-%m-%dT%H:%M:%S")
```
- This code sets a local script variable to the current timestamp at which the script is executed. Timestamps will be generated using YYYY-MM-DDTHH:mm:SS formatting – e.g. “2023-02-21T19:45:32”.
- You do not need to change any of this code, just copy and paste it directly.

Next, the second to last line (starting with “sh $CopyStormRootDir”) will need to be modified to write to the logfile. The final line will look like this:

sh $CopyStormRootDir/CopyStorm.sh -run $CopyStormConfigFile >> "$LOGDIR/CopyStorm_Log_$TIMESTAMP.txt" 2>&1

This code creates a logfile and redirects stderr and stdout to the file, capturing all job execution logs.

The example above will create log files with names like:

CopyStorm_Log_2023-02-21T19:45:32.txt

Here is an edited version of the batch script shown above:

Copy to Clipboard