Timing Data
Debugging where jobs spend most of their time can be useful in optimizing your pipeline, along with debugging spikes in job completion time.
Example
Generally, logging timing data is up to the command that is being run at the time, but we can manually add this by using some bash scripting.
See the following example:
- run: example_command | while read line; do echo "$(date +%T) > $line"; done;
This will provide an output along the following lines:
21:15:46 > Some Output Line 1 21:16:12 > Some Output Line 2 21:17:41 > Some Output Line 3
The format of the timing data is in HH:MM:SS, which provides granular information that will help track down steps that are taking longer than expected.
Comments
Article is closed for comments.