Scripts that do real work

Before you read. A script that archives log files has run every night for a year. Somebody adds a directory called Q3 reports, and the next morning the archive contains two entries: Q3 and reports. Neither exists.

Nothing changed in the script. The filename is legal, spaces have always been allowed in Unix filenames, and so have newlines, quotes, and leading hyphens.

The script was always broken. What made it look correct for a year was the data.

That is the shape of nearly every bug in this lesson. The script is not wrong in a way you can see by reading it; it is wrong in a way that only appears when the input stops being convenient.

This lesson is the set of tools that make a script robust rather than merely working: functions to give it structure, parameter expansion so it stops shelling out for string operations, safe iteration, real argument parsing, and cleanup that happens even when things go wrong.

Some words you will need

function

A named block of commands, callable like any other command, with its own arguments.

local

A variable that exists only inside the function that declared it.

parameter expansion

The ${...} forms that manipulate a variable's value without calling another program.

IFS

Internal field separator. What the shell splits unquoted values on.

trap

A command to run when the script receives a signal or exits.

idempotent

Safe to run twice. Running it again does not change the result.

What breaks without this

The script works on your data and fails on real data. Spaces, newlines, and leading hyphens in filenames are all legal, and every one of them breaks the naive loop.

Temporary files accumulate. A script that cleans up on its last line does not clean up when it fails on the line before, so /tmp fills with directories from runs that went wrong.

Nobody else can change it. A four-hundred-line script with no functions and no argument parsing is one people work around rather than modify.

And it half-runs. Interrupted at the wrong moment, it leaves a lock file, a partial archive, or a service stopped, and the next run refuses to start.

The filename problem

Here is the bug from the opening, exactly.

The directory contains three files, two of which have a space in the name. The script loops over $(ls reports), which is a common and entirely broken idiom.

# Debian 13 (trixie), x86_64
$ ls reports; echo "--- the script ---"; cat tidy.sh; echo "--- run it ---"; ./tidy.sh
q1 sales.txt
q2 sales.txt
simple.txt
--- the script ---
#!/bin/bash
for f in $(ls reports); do
    echo "processing [$f]"
done
--- run it ---
processing [q1]
processing [sales.txt]
processing [q2]
processing [sales.txt]
processing [simple.txt]

Five iterations for three files. The command substitution produced a string, the shell split it on whitespace, and each fragment became a separate item. Note that sales.txt appears twice and neither occurrence refers to a file that exists.

ls is not the problem, it printed the three names correctly, one per line, as the first three lines show. The problem is that command substitution discards that structure. The shell receives one flat string and has no way to know that the space in q1 sales.txt is part of a name while the newline after it is a separator; it splits on both, because both are whitespace.

The fix is to let the shell produce the list itself, using a glob:

# Debian 13 (trixie), x86_64
$ cat tidy-fixed.sh; echo "--- run it ---"; ./tidy-fixed.sh
#!/bin/bash
for f in reports/*; do
    echo "processing [$f]"
done
--- run it ---
processing [reports/q1 sales.txt]
processing [reports/q2 sales.txt]
processing [reports/simple.txt]

Three iterations, correct names. A glob expands to a list of words rather than a string, so no splitting is needed and none happens. This is the single most important habit in shell scripting: for f in * is correct and for f in $(ls) is not, and the difference is invisible until it is expensive.

For anything more selective than a glob, find -print0 is the answer:

# Debian 13 (trixie), x86_64
$ cat walk.sh; echo "--- run it ---"; ./walk.sh
#!/bin/bash
find reports -type f -print0 |
while IFS= read -r -d '' f; do
    echo "processing [$f]"
done
--- run it ---
processing [reports/q1 sales.txt]
processing [reports/q2 sales.txt]
processing [reports/simple.txt]

-print0 separates names with a null byte instead of a newline, and read -d '' reads up to a null. That combination is the only one that is correct for every legal filename, because null is the one byte a filename cannot contain. A newline can, and a script that splits on newlines is still broken, just more rarely.

rm -- *
rm ./*

Functions

A function is a named block of commands. It gets its own $1, $2, and $@, and it returns an exit status like anything else.

# Debian 13 (trixie), x86_64
$ cat report.sh
#!/bin/bash
set -euo pipefail

log() {
    printf '%s [%s] %s\n' "$(date +%H:%M:%S)" "$1" "$2"
}

usage_for() {
    local path="$1"
    local pct
    pct=$(df --output=pcent "$path" | tail -1 | tr -dc '0-9')
    echo "$pct"
}

main() {
    local target="${1:-/}"
    log INFO "checking $target"
    local used
    used=$(usage_for "$target")
    if [ "$used" -ge 90 ]; then
        log WARN "$target is ${used}% full"
        return 1
    fi
    log INFO "$target is ${used}% full, fine"
}

main "$@"

Several things there are deliberate and worth naming.

local on every variable inside a function. Without it, variables are global, so a function that uses i as a counter silently destroys the caller’s i. That is the most common function bug in shell and it produces symptoms far from the cause.

Declared and assigned on separate lines in usage_for:

local pct
pct=$(df ... )

That looks fussy and prevents a real problem. local pct=$(command) masks the command’s exit status, because local itself succeeds, so a failure inside the substitution is invisible, including to set -e. Splitting the two lines lets the status through.

A function returns a status, it does not return a value. return 1 sets the exit status; to hand back data you echo it and the caller captures with $( ), which is what usage_for does. return only accepts 0 to 255, so returning a percentage that way would work by accident and break at 256.

main "$@" at the bottom is a convention worth adopting. It means every statement in the file is a definition until the last line, so the script can be sourced for testing without doing anything, and the reading order matches the calling order.

# Debian 13 (trixie), x86_64
$ ./report.sh /; echo "rc=$?"
22:20:32 [INFO] checking /
22:20:32 [INFO] / is 7% full, fine
rc=0

Parameter expansion

Every one of these is a string operation the shell does itself, with no process started. They replace a surprising amount of sed, cut, basename, and dirname.

# Debian 13 (trixie), x86_64
$ cat expand.sh; echo "--- run it ---"; ./expand.sh
#!/bin/bash
name="${1:-world}"
file="report.2026.tar.gz"
path="/var/log/nginx/access.log"

echo "default if unset:  ${name}"
echo "length:            ${#file}"
echo "strip last ext:    ${file%.gz}"
echo "strip all ext:     ${file%%.*}"
echo "basename:          ${path##*/}"
echo "dirname:           ${path%/*}"
echo "replace all:       ${file//./-}"
--- run it ---
default if unset:  world
length:            18
strip last ext:    report.2026.tar
strip all ext:     report
basename:          access.log
dirname:           /var/log/nginx
replace all:       report-2026-tar-gz

The pattern is consistent once you see it: % works from the end, # from the beginning, and doubling the character makes the match greedy.

${path##*/} is basename and ${path%/*} is dirname, without forking. That matters in a loop over ten thousand files, where two processes per iteration is twenty thousand processes.

${v:?message} deserves more use than it gets. It is a one-line required- argument check:

target="${1:?usage: backup.sh SOURCE}"

If $1 is missing the script exits with that message and a non-zero status, which is three lines of if compressed into one.

log() { printf '%s [%s] %s\n' "$(date -Is)" "$1" "$2" >&2; }

if [ -t 1 ]; then
    red=$'\033[31m'; reset=$'\033[0m'
else
    red=''; reset=''
fi

logger -t myjob -p user.warning "disk at 91% on /var"

Parsing options properly

Hand-rolled argument parsing works until somebody combines flags or puts them in a different order. getopts handles the standard conventions:

while getopts ":vo:" opt; do
    case "$opt" in
        v) verbose=1 ;;
        o) output="$OPTARG" ;;
        \?) echo "unknown option: -$OPTARG" >&2; exit 2 ;;
        :)  echo "-$OPTARG needs a value" >&2; exit 2 ;;
    esac
done
shift $((OPTIND - 1))

The option string ":vo:" is dense and every character matters. A leading colon turns on silent error handling, which is what lets you write your own messages in the \? and : cases. A colon after a letter means that option takes a value, which arrives in $OPTARG.

shift $((OPTIND - 1)) is not optional. getopts sets OPTIND to the index of the first non-option argument, and the shift discards everything it consumed so that $1 becomes the first real argument. Forgetting it means "$@" still contains the flags.

# Debian 13 (trixie), x86_64
$ ./opts.sh -v -o /tmp/out.txt file1 file2; ./opts.sh -o; echo "rc=$?"; ./opts.sh -z; echo "rc=$?"
verbose=1 output=/tmp/out.txt remaining=file1 file2
-o needs a value
rc=2
unknown option: -z
rc=2

Both error cases are distinguished and both exit 2, which is the “used incorrectly” convention from lesson 51. That is what getopts buys over hand-parsing: the two failure modes are separate cases rather than one catch-all.

getopts handles short options only. It accepts -vo file combined, and it does not do --verbose. For long options the choices are GNU getopt (a different, external program with awkward quoting) or a manual while loop over case "$1". For a script you own, short options and a --help are usually enough.

exec 9>/var/lock/myjob.lock
flock -n 9 || { echo "already running" >&2; exit 0; }

tmp=$(mktemp "${target}.XXXXXX")
generate_content > "$tmp"
mv -- "$tmp" "$target"

Cleaning up whatever happens

A script that removes its temporary directory on the last line does so only when it reaches the last line. trap attaches cleanup to the exit instead.

# Debian 13 (trixie), x86_64
$ cat cleanup.sh; echo "--- run it ---"; ./cleanup.sh; echo "rc=$?"
#!/bin/bash
set -euo pipefail
workdir=$(mktemp -d)
trap 'echo "cleaning up $workdir"; rm -rf "$workdir"' EXIT

echo "working in $workdir"
touch "$workdir/scratch"
ls "$workdir"
echo "about to fail on purpose"
false
echo "never reached"
--- run it ---
working in /tmp/tmp.5NZt9tEhO0
scratch
about to fail on purpose
cleaning up /tmp/tmp.5NZt9tEhO0
rc=1

The cleanup ran and the failure status survived. never reached did not print, because set -e stopped the script at false, and the EXIT trap fired anyway, because it fires on every exit path. The script still reported 1, so the caller knows it failed.

EXIT is the trap to use, and it covers more than it looks: normal completion, exit anywhere in the script, a failure under set -e, and termination by SIGTERM or SIGHUP. It does not cover SIGKILL, which nothing can.

mktemp -d and a trap belong together, as a two-line habit at the top of any script that needs scratch space. mktemp creates the directory with mode 700 and an unpredictable name, which also closes a symlink-attack class that /tmp/myscript.$$ leaves open.

One caveat worth knowing: a second trap ... EXIT replaces the first rather than adding to it. If a script needs several cleanups, put them in one function and trap that.

Across distributions

None of this varies by distribution, only by shell, which is the same portability question as the two previous lessons.

The substitution and case-conversion forms are the ones to watch, because they fail silently in dash rather than erroring. The expansion simply does not happen the way you expect. ${v%pattern} and ${v#pattern} are POSIX and safe.

Prove it

# The two parse checks
bash -n script.sh
dash -n script.sh

# Everything above, found automatically
shellcheck script.sh

# Does it survive an awkward filename
mkdir -p /tmp/t && touch /tmp/t/"a b.txt" /tmp/t/-rf && ./script.sh /tmp/t

# Is it idempotent
./script.sh && ./script.sh && echo "twice is fine"

# Does it clean up when interrupted
./script.sh & sleep 1; kill %1; ls /tmp | grep -c tmp.

The awkward-filename test is the one that finds real bugs, and it takes one touch. A directory containing a b.txt and -rf exercises both the splitting and the leading-hyphen failure in a single run.

What trips people up

1. for f in $(ls)

Command substitution flattens the list to a string and the shell splits it on whitespace. Three files become five iterations.

for f in * for a glob, find -print0 piped to while IFS= read -r -d '' for anything more selective.

2. Variables leaking out of functions

Without local, everything is global, so a function using i destroys the caller’s i. The symptom appears somewhere else entirely.

local on every variable, and local x on its own line before x=$(command) so the command’s status is not masked.

3. Argument list too long

grep pattern * in a directory with a hundred thousand files exceeds the kernel’s limit on the size of an argument list.

find . -name '*.log' -exec grep pattern {} + batches them, and xargs does the same. The + rather than \; is what makes it batch instead of forking per file.

4. Cleanup on the last line

It runs only if the script reaches the last line, which it does not when it fails.

trap 'rm -rf "$workdir"' EXIT, registered immediately after creating the directory.

5. Appending configuration on every run

echo "x" >> file in a script that runs nightly produces a file with three hundred copies of the line.

grep -qxF "x" file || echo "x" >> file.

6. local x=$(command) hiding a failure

local succeeds, so the substitution’s exit status is discarded and set -e never fires. Declare and assign on separate lines.

Work it through

A log-archiving script has run nightly for a year and this morning produced an archive missing two directories. The relevant part:

#!/bin/bash
cd /var/log/app
for d in $(ls); do
    tar -czf /backup/$d.tar.gz $d
    rm -rf $d
done

Reason it out before reading on. There are four faults, and one of them is much worse than the others.

$(ls) splits on whitespace, so a directory called Q3 reports becomes Q3 and reports. That is the reported symptom, and it is the least dangerous fault here.

rm -rf $d is unquoted, and this is the serious one. With d set to Q3, the tar fails, but rm -rf Q3 also fails harmlessly. Now consider a directory called old logs: the loop runs with d=old, and rm -rf old removes a directory called old if one exists. The script deletes something it never archived, and the archive that was supposed to protect it does not contain it.

No error checking between tar and rm. Even with the quoting fixed, tar failing (disk full, permission denied) is followed immediately by rm -rf on the data that was not archived. The two lines must be connected by their status.

cd unchecked, which is lesson 51’s fault: if /var/log/app does not exist, the loop runs in whatever directory cron started in.

Repaired:

#!/bin/bash
set -euo pipefail

cd /var/log/app || exit 1

for d in */; do
    d="${d%/}"
    if tar -czf "/backup/${d}.tar.gz" -- "$d"; then
        rm -rf -- "$d"
    else
        echo "archive failed for $d, keeping it" >&2
    fi
done

*/ matches directories only and needs no ls. ${d%/} strips the trailing slash the glob adds. -- protects against a directory named like an option. And the if makes the deletion conditional on the archive succeeding, which is the fault that actually loses data.

The point worth extracting: three of the four faults were invisible for a year because the data was convenient. The script did not become wrong when somebody created Q3 reports; it was always wrong, and the directory name was the first input that exercised it. Testing with a deliberately awkward filename (one space, one leading hyphen) takes one touch and finds all of this before production does.

Try it

Optional, and use a scratch directory.

1. mkdir -p /tmp/t/"a b" /tmp/t/c && cd /tmp/t.
2. for f in $(ls); do echo "[$f]"; done and count the lines.
3. for f in *; do echo "[$f]"; done and count again.
4. touch -- -rf in that directory, then run ls * and see what happens. Then ls -- *.
5. Write a function that sets a variable without local, call it, and print the variable afterwards.
6. Add local and repeat.
7. Write a script that does workdir=$(mktemp -d), traps EXIT to remove it, and then calls false. Confirm the directory is gone and the status is non-zero.
8. shellcheck everything you wrote.

Verification step. You have it when a directory containing a b.txt and a file called -rf runs through your script with no surprises.

Check yourself

for f in reports/*; do ...            # a glob expands to a list, not a string
find reports -type f -print0 | while IFS= read -r -d '' f; do ...

local pct=$(df --output=pcent / | tail -1 | tr -dc '0-9')

local pct
pct=$(df --output=pcent / | tail -1 | tr -dc '0-9')

workdir=$(mktemp -d)
trap 'rm -rf "$workdir"' EXIT

grep -qxF "setting = value" /etc/app.conf || echo "setting = value" >> /etc/app.conf

References

• bash(1) - Linux man-pages project. Accessed 2026-08-08.
• find(1) - Linux man-pages project. Accessed 2026-08-08.
• xargs(1) - Linux man-pages project. Accessed 2026-08-08.
• mktemp(1) - Linux man-pages project. Accessed 2026-08-08.
• signal(7) - Linux man-pages project. Accessed 2026-08-08.
• Shell Command Language - The Open Group. Accessed 2026-08-08.

Every block above with a distribution and architecture header was captured by running the command on a Debian 13 (trixie) container. Blocks without one are illustrative.