I’m in the process of transferring data from an old Linux machine to a new one, and one particular directory contained too much data to compress into one single huge file, so I decided to look into ways of doing it smaller chunks.
The files that existed were mostly compressed anyway, so I decided to just do each of the directories into their own compressed file – but the thought of doing 75 directories “manually” made my heart sink.
Normally, to gather up one directory into a compressed “tarball” to save space and bandwidth, I would do:
tar zcvf newfile.tar.gz directoryname
Where tar (short for Tape ARchive) is the software that we use, z is the switch to tell Tar to use Gzip compression, c to create a new archive, v for verbose and f to create the file on the hard drive.
But as I said earlier doing that 75 times is enough to make me want to kick the hard drive across the room…
So, having a quick chat with a friend of mine via IRC, we discussed a couple of options and with what he gave me, I came up with the following:
find . -maxdepth 1 -type d -not \( -name "." \) -exec tar zcf \{\}.tgz \{\} \;
Using the find utility with the -maxdepth 1 (numeral one) and -type d options lists only the directories in the current directory, but only the top level (find will not recursively display directories).
We can then use the names as they are listed to create the archives.
Using -not \( -name “.” \) stops find from listing “.”, which in the Linux/Unix command line represents the current directory. If we didn’t do this, the whole of the current directory would also be processed, as well as the individual directories.
Lastly, -exec tar zcf \{\}.tgz \{\} \; will execute the tar commands where the curly brackets {} will represent the directory and file names. So if you have a directory called toodles, it will create a gzip’d tarball called toodles.tgz
This is just one of several ways it could probably have been done – it may look complicated to the uninitiated, but it did save me heaps of time and took less than 10 minutes in all.
You could also use “zip” to create Windows compatible archives, but these rarely compress down as much as Gzip does, and most decent Windows archive handlers (like Winrar) supports compressed tarballs.
There is also an option of using the j switch instead of z with tar – this will make tar use Bzip2 compression instead of Gzip. Bzip2 will compress files down even more than Gzip, however on some files and/or computers, it can take considerably longer to complete the task.