search
Ctrlk
Contact Support

How To Capture CopyStorm Runtime Logs

This article provides information on how to capture CopyStorm job execution log data that is output via Standard Error/Out in a text file. This approach is useful for doing advanced troubleshooting in order to capture error symptoms that may not appear within the application GUI or within the logged exceptions of the job history entry found on the “History” tab.

The general approach is as follows:

  1. Open a headless CopyStorm script for editing.

    1. This is the script that is scheduled using CRON or Windows Task Scheduler, and is created using the “Batch File” tab of CopyStorm.

  2. Create a directory that will be used to save the application logs.

  3. Edit the batch script to create a log file with the appropriate name in the established directory and write all job execution output to the file.

hashtag
Windows

Here is an unedited example of a Windows batch script generated by CopyStorm. This script will be modified to output CopyStorm job logs to a file:

@ECHO OFF
REM 
REM This script runs CopyStorm and relies on the notification system built into CopyStorm
REM to report the success/failure of a job. Log file information is written to stdout 
REM in addition to the logging tables in the CopyStorm database.
REM 
REM    CopyStormRootDir - the directory in which CopyStorm is installed.
REM    CopyStormConfigFile -- the name of the CopyStorm configuration file to run.

SET CopyStormRootDir=C:\Capstorm\CopyStorm
SET CopyStormConfigFile=C:\Capstorm\CapstormConfig\config.copyStorm

SET CopyStormCmd="%CopyStormRootDir%\CopyStorm.bat" -run "%CopyStormConfigFile%" 
CALL %CopyStormCmd% 2>&1

EXIT /B %ERRORLEVEL%

The following code is added after the line that begins with “SET CopyStormConfigFile”

    • This code sets a local script variable that establishes the path to the directory where the log files will be stored.

    • The “C:\Capstorm\CopyStormLogs” path should point to your desired CopyStorm log directory path.

    • This code sets a local script variable to the current timestamp at which the script is executed. Timestamps will be generated using YYYY-MM-DDTHH-mm-SS.SS formatting – e.g. “2023-02-21T19-45-32.12”.

    • You do not need to change any of this code, just copy and paste it directly.

Next, the second to last line (starting with “CALL %CopyStormCmd%”) will need to be modified to write to the logfile. The final line will look like this:

This code creates a logfile and redirects stderr and stdout to the file, capturing all job execution logs.

The “CopyStorm_Log_” prefix is used to identify the type of logfile – if you have multiple scheduled jobs, it is best practice to use a different prefix for each scheduled job. For example, “CopyStorm_Hourly_Data_” and “CopyStorm_Daily_Metadata_” if there is an hourly job for data and a separate daily job for metadata.

The example above will create log files with names like:

  • CopyStorm_Log_2023-02-21T19-45-32.txt

Here is an edited version of the batch script shown above:

hashtag
Linux

Here is an unedited example of a Linux sh script generated by CopyStorm. This script will be modified to output CopyStorm job logs to a file:

The following code is added after the line that begins with “CopyStormConfigFile=”

    • This code sets a local script variable that establishes the path to the directory where the log files will be stored.

    • The “/opt/Capstorm/CopyStormLogs” path should point to your desired CopyStorm log directory path.

    • This code sets a local script variable to the current timestamp at which the script is executed. Timestamps will be generated using YYYY-MM-DDTHH:mm:SS formatting – e.g. “2023-02-21T19:45:32”.

    • You do not need to change any of this code, just copy and paste it directly.

Next, the second to last line (starting with “sh $CopyStormRootDir”) will need to be modified to write to the logfile. The final line will look like this:

This code creates a logfile and redirects stderr and stdout to the file, capturing all job execution logs.

The “CopyStorm_Log_” prefix is used to identify the type of logfile – if you have multiple scheduled jobs, it is best practice to use a different prefix for each scheduled job. For example, “CopyStorm_Hourly_Data_” and “CopyStorm_Daily_Metadata_” if there is an hourly job for data and a separate daily job for metadata.

The example above will create log files with names like:

  • CopyStorm_Log_2023-02-21T19:45:32.txt

Here is an edited version of the batch script shown above:

Last updated

Was this helpful?