Systems Administration

In a previous post, I discuss a couple of approaches to dealing with the "Argument list too long" error when transferring large numbers of files. The solution to this problem is to archive the files, using the "-T" option of the "tar" command to pass in a list files generated by a "find" command:

1. Create a list of the files to be archived using the "find" command:

   $ find . -name="*.tre" > filelist.txt

2. Use the "-T" option of the "tar" command to pass in this list of filenames:

   $ tar cvjf archive.tbz -T filelist.txt

If you want to delete a long list of files, however, this approach will not work, as "rm" does not support the very convenient "-T"/"--files-from" flag or the equivalent (so convenient, in fact, that I have started adding this to virtually every file-processing script or program that I write).

Luckily, however, "find" does support a "-delete" flag, so to recursively delete all files and directories:

 

You can get a "bird's eye" view of your cluster load by running:

We have all done it before. Idiotic actions that result in catastrophic results (usually in conjunction with "sudo"). Some more than others. Have you ever tried clearing out some directory using "sudo rm -rf *", and then realized you were actually in the root directory? What about "sudo rm -rf .*", in the mistaken impression all you are going to being is to blow away hidden files in the current directory?

A firewalled-server that restricts access only through a gateway machine is a reasonable security measure, but it is a pain to have to hop through the gateway everytime you need to transfer files. It also makes it difficult to use local applications to work directly on files on the firewalled server, even if the applications support ssh- and ssftp-based editing protocols. Using a FUSE (Filesystem in Userspace) approach, you can mount a remote directory on a firewalled-server as a local volume (or directory), resulting in seamless and hassle-free access.