Open Computing ``Hands-On'': ``Wizard's Grabbag'' Column: January 94

Getting a Bang Out of Unix

Automatically generating temporary files, saving money by sending e-mail in batches, and revisiting the rounding error in ap

With the start of the new year and the premiere of UnixWorld's Open Computing, we are taking this opportunity to make some improvements to ``Wizard's Grabbag.'' This month we begin a new experiment: selecting and presenting items from O'Reilly & Associates' Unix Power Tools, a book that presents tips, scripts, and techniques like those published in this column. From time to time I'll publish interesting items useful to most readers and try to locate related material that didn't make it into the book. For our first selection we present a script that automatically generates temporary files for command operations requiring file arguments.

In the ``From the Wizard's Den'' section I provide a tip that can help some site administrators configure their system to save money by batching outgoing electronic mail.

Finally, we have ``Closing the Feedback Loop,'' which contains selected contributions related to recent columns. Harvey Davies provides some feedback regarding his ap program that was originally published in my August 1992 column.

Bang Around the Command Line

Dear Editor:

Here's a little tool that should prove useful to anyone who works from the shell command line. Some commands or command modes require you to provide an argument that names a file instead of supplying that data on the standard input or the command line. Such files are often created just for the command operation and then erased. The !, or ``bang,'' script [Listing 1A] can be used to name a temporary file that is populated by running a specified command.

I'll start with an example that can be simplified with ! [Listing 1B]. Here, we manually create two temporary files that contain sorted versions of the input data. With the comm program, we compare the contents of these two sorted files. The default comm report--no invocation options specified--consists of three columns: unique items in the first file, the second file, and common to both files. The next example [Listing 1C] shows how to collapse the four command lines [used in Listing 1B] into a single command using !.

We use ! to expand tab stops in files before they are compared by diff [shown in Listing 1D]. The diff output from files containing tabs may appear misaligned because of the extra characters (>, <, +, and so forth) that diff prefixes to the output line. If you expand the tabs to spaces before using diff, proper alignment of the output can be maintained.

Some popular paginators like more can't back up when they're reading from a pipe. Perusing the output from a pipe can be painful if you can't reverse direction to reread something you saw earlier. For instance, ls -l | more displays a screenful at a time from ls - l output, but if something of interest has scrolled past, you can't review it unless, of course, you're using a display system with reverse- scroll capability. However, if you invoked more using more `! ls - l`, you can review the result as much as you wish by moving around the temporary file with more file- perusal commands.

The main problem with the ! script published in Unix Power Tools was that it didn't remove the temporary file it created. The script can't remove this file as it exits because then the temporary file might not be found when needed by the ``driver'' command [like comm in Listing 1C]. The new version in Listing 1A waits 30 seconds for the ``driver'' command to start and then erases the temporary file. Your readers could take this approach farther by adding an option that changes the delay time. For instance, ! -600 would wait 10 minutes (600 seconds) and ! -0 would never remove the file.

Explanation of ! script. Line 7 defines the parent directory, and line 9 defines the full path name of the unique temporary file. Here, $$ will be replaced by the process ID number of the shell running the instance of the ! script.

Line 12 sets a trap to erase the temporary file when the script is terminated by one of the specified signals: hangup (1), interrupt (2), or software termination (15). The quit (3) signal is not trapped; it will abort ! without removing the temporary file.

Lines 15-19 process the command line. If no invocation arguments were specified, line 16 reports the correct usage to standard error, line 17 outputs the path name of the temporary file, and the script terminates. Hopefully, when the driver command processes the (missing) temporary file, it will exit gracefully.

Line 22 runs the specified command line, which is contained in the $@ variable, and redirects its output to the temporary file. Line 23 reports the temporary file name on the standard output for use by the driver command.

Line 26 is important. This line closes the standard output so the shell that's ``filling the backquotes'' with the output of the ! program won't wait for more output. Without this line, the calling shell would wait until the subshell on line 27 exits and closes its standard output.

Line 27 runs a subshell in the background to erase the temporary file after 30 seconds.

Big Bang Contest!

We're starting a new contest this month. Send in your shell command line examples for using the ! program discussed here. The most clever or useful contributions will be featured in a future column.

From the Wizard's Den: Batch UUCP Mail and Save

When Mail User Agents (MUAs) such as mailx realize that a message is destined for a remote machine connected via UUCP, it invokes uux to queue a request to run the rmail command on the next machine. The rmail command will deliver the mail message to a subsequent machine or place it in the mailbox for the recipient if the next hop is the final destination machine.

When uux finishes queuing the request, it attempts to contact the next machine by invoking uucico. Unfortunately, the time it takes to establish and sever a UUCP connection to a remote machine often takes longer than sending the message itself! Thus, it would be more efficient to collect all mail requests during an appropriate time interval and then send them all at once. To do this, we need to have the mail program run uux with the -r option, which will queue the request but not invoke uucico.

If you can't configure your MUA to run uux -r instead of uux, you could install a shell script named uux, which in turn would run the actual uux binary with the -r option. Listing 2A shows one way to set up this configuration.

We change to the directory containing the uux binary. Then we rename the binary to something else; UUX is a convenient choice. Next we create a simple one-line shell script containing the directive to run the UUX binary with the -r option, followed by any arguments passed to the script. We give the script 755 permission modes so it's executable by anyone on the system. Finally, we check our work with a long directory listing.

The queued mail will be sent when uucico is invoked. Generally, the cron daemon runs an ``hourly'' script-named /usr/lib/uucp/uudemon.hour with HoneyDanBer UUCP--periodically, say once or twice an hour. This script in turn invokes uusched, which will find any queued mail messages and invoke uucico to contact the next machine on their way to their final destination.

Now that we have a shell script that's invoked to run UUX , we can add other useful auditing functions. For instance, we can log the actual command line to see where the message is destined. This feature is provided automatically with System V Release 4 HoneyDanBer UUCP, which logs all uux as well as uucp invocations in a file named /var/spool/uucp/.Admin/command.

I've found it useful to display the environment of the shell running the uux script. In particular, if mail is passing through our site, uux will be invoked by the ``uucp'' user. Examine the UU_USER environment variable for this user to see where the e-mail originates. I've found this tactic useful for tracing excessive e-mail destined for users who no longer use our site yet still receive mail forwarded by our site, such as automatic messages from mailing lists. Also, I like to display the complete shell environment-by running the System env or BSD printenv command--for system accounts that run commands using cron, such as ``root'', ``adm'', and ``sys''. If any of our readers find any other interesting applications, let me know.

Listing 2B shows a sample script that supports the additional features I've described. Caution: Make sure the script works correctly before you replace the uux binary. Otherwise, a syntax error could cause the script to abort so the UUX binary would not be run. Also, some commands that you call from the uux script might have unanticipated side effects. For instance, if you include a command to mail an error report to a remote user, uux could be called recursively until some system resource is exhausted, such as all the free blocks or inodes in the file system containing the UUCP spooling directory.

Listing 2C shows some lines to use in the uudemon.cleanup script to mail the uux log to the UUCP administrator and then shuffle the log, keeping one backup copy. We copy instead of moving the log to the Old directory so the subsequent step will truncate the original copy without changing its ownership or permission modes. All uux users will need to write to this log--but they don't need to read it--so you could install it as owned by the ``root'' or ``uucp'' account with mode 622.

Closing the Feedback Loop: Correcting the Correction

Dear Editor:

Your August 1992 column featured my ap program, and you published a ``bug'' fix in February 1993. However, I wish to point out a rounding-error problem caused by the modification proposed by David Bacon. For instance, the command line ap -n 10 0 3.5 0.14 run on a machine using IEEE floating-point arithmetic displays a sequence that omits the expected final value of 3.5 [see Listing 3A].

Mr. Bacon suggested that you change the constant on line 90 of the original program from 1.5 to 1.0 [see Listing 3B]. My value of 1.5 minimizes the effect of rounding error, but does allow a final value beyond a specified limit that does not correspond to an element of the arithmetic progression. For instance, the ap 1 6 2 command prints ``1 3 5 7'' with the original version, and the modified version prints ``1 3 5''.

My original letter should have pointed out the need to specify a limit within less than half a step of the final value to be produced. I do not believe this limit is a significant problem, and as far as I can see, any attempt to solve it will increase the possibility of the more serious omission problem discussed above. Furthermore, you could add code to not display values greater than the limiting value.

My other suggestion is to make the default number of elements per line infinity--rather than 50--because the program is mainly used with command substitution, as in contour -h `ap -s , - n 999 0 100`. This change eliminates the need to specify the number of elements per line in such cases [see Listing 3C].

WANTED: Alternate ap Implementations

Here's a challenge for our programming and numerical analysis buffs: Provide an alternate implementation of ap that doesn't have the ``rounding-error'' problem discussed herein. One suggestion: instead of computing the number of elements to display, display elements until the limiting value is reached or exceeded. Of course, you would need to accommodate both positive and negative integer and floating-point values for the starting, limiting, and step values. Also, I'd recommend support for a repeat value using a command-line option-like -r count--instead of a special case for input values (step is zero).