Estimate Time for Job Completion (With Progress Updates) When Tar'ing Huge Directories

For the sake of future me, I am recording this here, the coolest shell trick I’ve learned this year: (Linux):

tar cf - /folder-with-big-files -P | pv -s $(du -sb /folder-with-big-files | awk '') | gzip > big-files.tar.gz

(OSX):

tar cf - /folder-with-big-files -P | pv -s $(($(du -sk /folder-with-big-files | awk '') * 1024)) | gzip > big-files.tar.gz

with output looking like:

4.69GB 0:04:50 [16.3MB/s] [==========================>        ] 78% ETA 0:01:21

Requires ‘pv’: https://github.com/icetee/pv

Reproduced from this Superuser answer here: https://superuser.com/a/665181

Share