How to optimize the restore_cache step for Node projects

The two main operations in the restore_cache step are: "download" and "unarchive".

In the case of Node related cache, we found that the "unarchive" operation can sometimes take a significant amount of time, and that the duration can be inconsistent across builds.

 

Disk I/O contention

The issue described above is mostly due to disk I/O contention occurring, and being aggravated by the structure of Node dependencies.

Often "node_modules" folders will contain a huge number of small files, and restoring all these files can cause the aforementioned resource contention, and can lead to a high deviation in the restore_cache duration.

 

RAM disk

To improve consistency of the restore_cache step's duration, we suggest using the RAM disk available for each container.

Note that depending on the jobs' memory usage, you might need to use/specify a larger resource class to ensure there is enough memory to leverage the RAM disk feature.

 

Additional information

If you'd like to learn more about this topic, consider reading this blog post in which we provide a detailed analysis of this behaviour.

Was this article helpful?
0 out of 0 found this helpful