Shell script common template

Question

Millions of developers write shell scripts to solve various types of tasks. I use shell scripts to simplify deployment, life-cycle management, installation or simply as a gl

Answer 1

I will also share my results. The idea behind all these examples is to encourage overall quality. It is also important to make sure final result is safe enough.

Logging

It is really crucial to have proper logging available from the same beginning. I'm just trying to think about production usage.

TAG="foo"
LOG_FILE="example.log"

function log() {
    if [ $HIDE_LOG ]; then
        echo -e "[$TAG] $@" >> $LOG_FILE
    else
        echo "[`date +"%Y/%m/%d:%H:%M:%S %z"`] [$TAG] $@" | tee -a $LOG_FILE
    fi
}

log "[I] service start"
log "[D] debug message"

Command test

This is about safety, real-life environments and proper error-handling. Could be optional.

function is_command () {
    log "[I] check if commad $1 exists"
    type "$1" &> /dev/null ;
}

CMD=zip

if is_command ${CMD} ; then
   log "[I] '${CMD}' command found"
else
   log "[E] '${CMD}' command not found"
fi

Template processing

Could be only my subjective opinion, but anyway. I used several different ways to generate some configuration/etc right from the script. Perl, sed and others do the job, but look a little bit scary.

Recently I noticed a better way:

function process_template() {
    source $1 > $2

    result=$?
    if [ $result -ne 0 ]; then
        log "[E] Error during template processing: '$1' > '$2'"
    fi
    return $result
}

VALUE1="tmpl-value-1"
VALUE2="tmpl-value-2"
VALUE3="tmpl-value-3"

process_template template.tmpl template.result

Template example

echo "Line1: ${VALUE1}
Line2: ${VALUE2}
Line3: ${VALUE3}"

Result example

Line1: tmpl-value-1
Line2: tmpl-value-2
Line3: tmpl-value-3

Answer 2

I would steer clear of relying on bash as the shell and model your solution on top of shell syntax defined by POSIX and use /bin/sh on the shebang. We had a number of surprises recently when Ubuntu changed /bin/sh to dash.

Another pandemic in the shell world is a general misunderstanding of exit status codes. Exiting with an understandable code is what lets other shell scripts programmatically react to specific failures. Unfortunately, there is not a lot of guidance on this beyond the "sysexits.h" header file.

If you are looking for more information about good shell scripting practices, concentrate on Korn shell scripting resources. Ksh programming tends to focus on really programming as opposed to writing haphazard scripts.

Personally, I haven't found much use for shell templates. The unfortunate truth is that most engineers will simply copy and paste your template and continue to write the same sloppy shell code. A better approach is to create a library of shell functions with well-defined semantics and then convince others to use them. This approach will also help with change control. For example, if you find a defect in a template, then every script that was based on it is broken and would require modifications. Using a library makes it possible to fix defects in one place.

Welcome to the world of shell scripting. Writing shell scripts is a bit of a lost art that seems to be entering a renaissance. There were some good books written on the subject in the late 90's - UNIX Shell Programming by Burns and Arthur comes to mind though the Amazon reviews for the book make it seem awful. IMHO, effective shell code embraces the UNIX philosophy as described by Eric S. Raymond in The Art of Unix Programming.

Answer 3

This is the header of my script shell template (which can be found here: http://www.uxora.com/unix/shell-script/18-shell-script-template).

It is a man look alike which is used to by usage() to diplsay help as well.

#!/bin/ksh
#================================================================
# HEADER
#================================================================
#% SYNOPSIS
#+    ${SCRIPT_NAME} [-hv] [-o[file]] args ...
#%
#% DESCRIPTION
#%    This is a script template
#%    to start any good shell script.
#%
#% OPTIONS
#%    -o [file], --output=[file]    Set log file (default=/dev/null)
#%                                  use DEFAULT keyword to autoname file
#%                                  The default value is /dev/null.
#%    -t, --timelog                 Add timestamp to log ("+%y/%m/%d@%H:%M:%S")
#%    -x, --ignorelock              Ignore if lock file exists
#%    -h, --help                    Print this help
#%    -v, --version                 Print script information
#%
#% EXAMPLES
#%    ${SCRIPT_NAME} -o DEFAULT arg1 arg2
#%
#================================================================
#- IMPLEMENTATION
#-    version         ${SCRIPT_NAME} (www.uxora.com) 0.0.4
#-    author          Michel VONGVILAY
#-    copyright       Copyright (c) http://www.uxora.com
#-    license         GNU General Public License
#-    script_id       12345
#-
#================================================================
#  HISTORY
#     2015/03/01 : mvongvilay : Script creation
#     2015/04/01 : mvongvilay : Add long options and improvements
# 
#================================================================
#  DEBUG OPTION
#    set -n  # Uncomment to check your syntax, without execution.
#    set -x  # Uncomment to debug this shell script
#
#================================================================
# END_OF_HEADER
#================================================================

And here is the usage functions to go with:

  #== needed variables ==#
SCRIPT_HEADSIZE=$(head -200 ${0} |grep -n "^# END_OF_HEADER" | cut -f1 -d:)
SCRIPT_NAME="$(basename ${0})"

  #== usage functions ==#
usage() { printf "Usage: "; head -${SCRIPT_HEADSIZE:-99} ${0} | grep -e "^#+" | sed -e "s/^#+[ ]*//g" -e "s/\${SCRIPT_NAME}/${SCRIPT_NAME}/g" ; }
usagefull() { head -${SCRIPT_HEADSIZE:-99} ${0} | grep -e "^#[%+-]" | sed -e "s/^#[%+-]//g" -e "s/\${SCRIPT_NAME}/${SCRIPT_NAME}/g" ; }
scriptinfo() { head -${SCRIPT_HEADSIZE:-99} ${0} | grep -e "^#-" | sed -e "s/^#-//g" -e "s/\${SCRIPT_NAME}/${SCRIPT_NAME}/g"; }

Here is what you should obtain:

# Display help
$ ./template.sh --help

    SYNOPSIS
    template.sh [-hv] [-o[file]] args ...

    DESCRIPTION
    This is a script template
    to start any good shell script.

    OPTIONS
    -o [file], --output=[file]    Set log file (default=/dev/null)
                                  use DEFAULT keyword to autoname file
                                  The default value is /dev/null.
    -t, --timelog                 Add timestamp to log ("+%y/%m/%d@%H:%M:%S")
    -x, --ignorelock              Ignore if lock file exists
    -h, --help                    Print this help
    -v, --version                 Print script information

    EXAMPLES
    template.sh -o DEFAULT arg1 arg2

    IMPLEMENTATION
    version         template.sh (www.uxora.com) 0.0.4
    author          Michel VONGVILAY
    copyright       Copyright (c) http://www.uxora.com
    license         GNU General Public License
    script_id       12345

# Display version info
$ ./template.sh -v

    IMPLEMENTATION
    version         template.sh (www.uxora.com) 0.0.4
    author          Michel VONGVILAY
    copyright       Copyright (c) http://www.uxora.com
    license         GNU General Public License
    script_id       12345

You can get the full script template here: http://www.uxora.com/unix/shell-script/18-shell-script-template

Answer 4

Here's my bash boilerplate with some sane options explained in the comments

#!/usr/bin/env bash

set -e  # Abort script at first error, when a command exits with non-zero status (except in until or while loops, if-tests, list constructs)
set -u  # Attempt to use undefined variable outputs error message, and forces an exit
set -x  # Similar to verbose mode (-v), but expands commands
set -o pipefail  # Causes a pipeline to return the exit status of the last command in the pipe that returned a non-zero return value.

Answer 5

If you're concerned about portability, do not use == in tests. Use = instead. Do not explicitly check if $# is 0. Instead, use ${n?error message} the first time you reference a required argument (eg ${3?error message}). This prevents the extremely annoying practice of emitting a usage statement instead of an error message. And most importantly, always put error messages on the right stream and exit with the correct status. For example:

echo "Unknown error while processing options" >&2
exit 1;

It is often convenient to do something like:

die() { echo "$*"; exit 1; } >&2

Answer 6

There is no more helpful thing to a shell script than a well documented behaviour with examples of usage and a list of known bugs. I think that no one program can be titled bulletproof, and bugs may appear in every moment (especially when your script gets used by the other people), so the only thing I’m taking care of is the good coding style and using only these things that the script really need. You’re standing on the way of aggregating, and it always going to become a large system which comes with a lot of unused modules, which is hard to port and hard to support. And the more system trying to be portable, the bigger it grows. Seriously, shell scripts do not need it to be implemented in that way. They must be kept as small as possible to simplify further use.

If the system really needs something big and bulletproof, it’s time to think about C99 or even C++.