Michael Paoli's Perl stuff

Just a slight sampling at the moment ...

Here's a backup utility (and helper scripts) I wrote for myself and found to be quite useful to me. These are some of its capabilities:

searches for and determines the dev=#,#,# for cdrecord for the CD-RW device I'm looking for
determines maximum write speed, based both on media capabilities and drive limitations
determines capacity of media
has quite flexible configuration for what's initially done and backed up (e.g. to capture key meta-data, such as disk configuration and information on filesystem sizes and logical volume manager configuration - typically the data one wants first when doing a "bare metal" restore - particularly if target may not precisely match original hardware)
backs up filesystems in the order I want, based on a somewhat more than trivial sort algorithm (most notably backing up in the order I'm most likely to need/want to restore if/when I need to restore everything from scratch)
relatively flexible/modular backup mechanism (e.g. pass some filesystem parameters to a helper script, which then outputs stream of data to be backed up for the filesystem)
spools the data to a device/file before writing to CD-R*/RW
writes spool efficiently, treating it logically as if it were a ring buffer
writes to CD-R*/RW in 2,048 byte blocks, padding with nulls if needed to 2,048 byte block boundary at end of track/CD
keeps relatively simplistic index track of what filesystem backups start on which media volume and at what 2,048 byte offset from start of volume, and includes the completed index information at the end on stdout and also to the media (since compression is typically used, the offsets aren't known ahead of time, hence the index is completed at the end)
handles various media types/states (blank or previously written CD-RW media, blank CD-R* media)
handles media write errors (that, and potential data underruns, were key motivators for the spool - if a write fails, one can try it again)
and probably a fair number of other things that didn't jump to mind as I wrote this web page

*I haven't tested it with CD-R media yet - haven't yet had need/motivation to use CD-R media for backups or test writes

Anyway, that Perl script and the related stuff (helper scripts and such) may be found in this gzipped tar bundle: backup,etc.tar.gz. Be sure to also review the bin/README.txt file in that bundle. There's also a fairly useful spattering of comments in at least the main Perl source file.

Here's a CoMPare and LiNk (cmpln) utility I wrote.
It's intended to be used on UNIX-like (UNIX, LINUX, BSD, ...) type filesystems, primarily to do things such as reduce reduncancy in filesystem archive collections. From its --verbose --help:
cmpln examines pathname(s) looking for distinct occurrences of non-zero length regular files on the same file system with identical data content in which case cmpln attempts to replace such occurrences with hard links to the occurrence having the oldest modification time, or if the modification times are identical, the occurrence having the largest number of links, or if the link count is also identical to an arbitrarily selected occurrence.

Some of the things this utility does and/or which made the task well suited for implementation in Perl:

much higher level and more feasible than implementing in C
would be infeasibly ugly and complex to attempt to implement in shell, and couldn't be nearly as efficient as with Perl
relatively easy ability to examine metadata about files to avoid data comparisons where files could not match, could not be linked, or where data comparisions would be redundant (already linked)
ability to sort (meta) data to handle any comparisions in an efficient order and manner
ability to quickly and easily discard unneded or no longer needed data (reducing program resource consumption)
relative ease of manipulating a relatively arbitrary number of open filehandles in data structures
efficient multi-way file comparision:
- files are read by specified blocksize
- files are never read more than once
- files are only read so long as a potential match still exists

At least thus far, it knows nothing about ACLs, so don't try to get too tricky on it. It's primarily intended for local UNIX(-like) filesystems, so it may not work or may not work well in substantially different environments. Anyway, it can be grabbed here: cmpln.tar.gz

disable_users - quite securely lock accounts out. Also has options to save and restore (reverse) most of the actions of disabling an account. The --help gives a pretty good overview of its functionality:

$ disable_users --help
usage: disable_users [-f|--force] [-h|-?|--help] [-n|--nochange] [-r directory|--restore directory] [-s directory|--save directory] [-u uid|--uid uid] [-v [level]|--verbose [level]] [login ...]
    -f|--force
        be forceful - bypass or relax some checks
    -h|-?|--help
        help - provide some basic usage information, overrides other options
        except verbose
    -n|--nochange
        no account change actions - do not alter accounts or signal PIDs
    -r directory|--restore directory
        restore - reverse most effects of disabling where --save was used
    -s directory|--save directory
        save data to directory so --restore may later be used
    -u uid|--uid uid
        specify uid, signals PIDs, may be given multiple times, does not
        itself trigger processing of any associated login(s)
    -v [level]|--verbose [level]
        be verbose, optionally specifying verbosity level
    login
        login name

Some of the funky invocation in the script is to deal with working around some flakey (broken) perl installations on systems where more functional installations of perl also exist.
Anyway, it can be grabbed here: disable_users.tar.gz

iso2epoch
Here's a quick utility I whipped to covert some common ISO (and ISO-like?) date/time formats to seconds since the epoch.
From iso2epoch --help:
usage: iso2epoch [-0] [--help|-h|-?] [file|timespec] ...
iso2epoch reads timespecs from command line argument(s) and/or files,
or standard input, if no non-option arguments are given,
and outputs seconds since the epoch.
-h|--help overrides all other options/arguments/input and provides usage
-0 null terminated/separated input and output (default is \n)
timespecs should be of the form:
YYYY-MM-DD(T| )HH:MM:SS(\.\d+|[-+]\d{2}(|:?\d{2}))?
non-option arguments matching the above are taken as timespecs,
otherwise they are taken as files.
iso2epoch.tar.gz

Last-Modified_touch - A simple utility I wrote to set the mtime on a file based upon the a Last-Modified header. Last-Modified_touch

mimencode - Simple utility I wrote to replace some of the functionality of mimencode(1) - notably that which I actually used, when my Linux distribution dropped the package that contained mimencode. It basically does some very simple MIME encoding/decoding, to/from base64 and quoted printable formats.

mimencode

multichown - works somewhat similar to chown(1), but handles complex non-overlapping multiple UID/GID remappings, and does so in a single pass. The primary design criteria has been do to UID/GID remappings where large numbers of UIDs/GIDs need to be aligned, e.g. to LDAP, on systems where the UID/GID ownerships aren't already suitably aligned, and to do so in an efficient manner (e.g. for large filesystems that take many hours to traverse, all the changes are made in a single pass). The --help gives a pretty good overview of its functionality:

$ multichown --help
usage: multichown [--force|-f] [--help|-h|-?] [--nochange|-n] [--noxdev|--nomount] [--recursive|-r|-R] [--verbose [level]|-v [level]] IDspec [IDspec ...] pathname [pathname ...]
    --force|-f
        Be forceful - bypass some checks/restrictions.
    --help|-h|-?
        Help - provide some basic usage information, overrides other options.
    --nochange|-n
        Change nothing, but say what would otherwise be done.
    --noxdev|--nomount
        Descend directories on other filesystems.
    --recursive|-r|-R
        Recursively descend directories.
    --verbose [level]|-v [level]
        Be verbose, optionally specifying verbosity level:
            -1 not verbose (default if --verbose not used)
             0 report items changed (and not changed if --nochange)
               (default if --verbose used without level)
             1 0 and argument processing and report items not changed
             2 1 and include diagnostics on options and items processed
             3 2 and more diagnostics on options
    IDspec - ID specification, specifcation of set of ID changes as follows:
        fromUID,toUID[,fromGID,toGID ...]
        each ID must be specified as a decimal integer, any UID owned by
        fromUID will be lchown()ed to be owned by toUID, additionally, if
        and as specified, if the owning GID is fromGID, it will have the
        group ownership changed to the immedately following specified
        toGID
    pathname - pathname(s) to examine

multisum - compute one or more hash sums on file(s) in a single read pass of file(s). The --help gives a pretty good overview of its functionality:

$ multisum --help
multisum [options] [file ...]
    Options are case insensitive, except as noted
    --help - print this help text and exit, overrides other options
    --blocksize blocksize
    --blocksize=blocksize
    --bs blocksize
    --bs=blocksize
        set input read blocksize to blocksize, default units in bytes,
        default blocksize is 4096 bytes, the following suffix multipliers
        may be used, those ending in iB are case sensitive:
        s     - 512 byte "blocks" (sectors)
        k KiB - 2*s
        m MiB - 1024 KiB
        g GiB - 1024 MiB
        t TiB - 1024 GiB
        p PiB - 1024 TiB
        e EiB - 1024 PiB
        z ZiB - 1024 EiB
        y YiB - 1024 ZiB
    algorithms to use, by default we use these:
    MD5 SHA-1 SHA-256 SHA-512
    algorithm(s) used may be specified by options below,
    specifying any of these overrides use of the default algorithm(s):
        --md5 --md5sum - MD5
        --sha-1 --sha-1sum --sha1 --sha1sum - SHA-1
        --sha-224 --sha-224sum --sha224 --sha224sum - SHA-224
        --sha-256 --sha-256sum --sha256 --sha256sum - SHA-256
        --sha-384 --sha-384sum --sha384 --sha384sum - SHA-384
        --sha-512 --sha-512sum --sha512 --sha512sum - SHA-512
        --all --allsum --allsums - use all of the above algorithms

Anyway, it can be grabbed here: multisum

A reformatting utility I wrote. Mostly attempts to semi-intelligently add missing spaces around punctuation, fixes some other bits of writing, and optionally also make case conversions. I mostly use(d) it to make more readable someone's writings that otherwise were a fair bit more difficult to read.
reformat.

$ reformat --help
usage: reformat [--help|-h|-?] [--case|-c] [file ...]
    --help|-h|-?
        Help - provide some basic usage information, overrides other
        options.
    --case|-c
        also apply case conversions
    file - file(s) to process (defaults to stdin)

Oakland Perl Mongers Oakland Perl Mongers