Andrew Kesterson

"Love God. Live Righteously. Die Well."
Source Code || LinkedIn
29 Posts 9 Categories 0 Tags
Categories
Books 14
Technology 11
Faith 6
Outdoors 1
Leadership 8
Current-Events 5
Philosophy 5
History 1
Liberal-Education 1
hot tags

# This Old Tool : cmdarg

2026-01-19 Technology 2.9k words

A craftsman tends to collect up a few tools over the course of their career that travel with them from job to job, because they need them on every job, and you really prefer having your tools, the ones that you’ve chosen for your reasons because they work well for you. Your hammer, your circular saw, your calculator, your IDE configuration. I recently had to do some maintenance on some of the oldest tools in my toolbox, so I thought now would be a good time to talk about those tools, and the classes of problems they solve. Because, despite how advanced our modern programming tools have become, I find myself having to solve some of hte same basic problems basically everywhere I go; and I have yet to find a solution that works better than the tools I wrote long ago. So, in a series of articles, I’ll be presenting some of the oldest, most well-worn, hand-made tools in my toolbox.

These are not the only tools available to solve these problems. I am not even necessarily saying they are the best (though I can make some strong arguments in at least one case). These are just the ones that I have written early in my career, and found useful enough to maintain, because I have had to use them in every single technology job I’ve ever worked since I first wrote them. Maybe someone else will find them useful, or will benefit from exploring my thought process or philosophy of usage.

## cmdarg: Parsing arguments on bash scripts

If you spend more than a few weeks administering any kind of Linux/UNIX system, whether a server or a desktop, you will find yourself writing bash scripts to extend and automate the administration or usage of that system. At first your scripts will be simple - a script that wraps a few commands with some hardcoded values in it. Then you’ll wind up having to share the script with someone else, or extend the script for some new use case. This will require that you variablize some of the code that used to have hardcoded values in them. Project IDs, hostnames, secrets, input values, whatever. Your script now has some kind of information that you need to take in from the user in order to do your job.

In bash, you have four ways (or, at least, four large stylistic buckets) for how to handle this:

- Move all of your variables to the top of your script, and tell the user to change them before running it
- Read environment variables for user configurable input
- Use the bash builtin `getopts` to parse arguments on the command line
- Use the program `getopt` to parse arguments on the command line

There are benefits and drawbacks to each approach.

- Variables at the top of the script is bulletproof from the script writer’s perspective, but very inconvenient from a user’s perspective
- Environment variables are nearly as bulletproof from the script writer’s perspective, and not all that inconvenient from the user’s perspective. However there is no kind of help message available at the command line, so to review the interface, you have to open the script. You’re also limited in the kind of values that can be easily transmitted this way - strings and integers are pretty much it.
- `getopts` is simple and guaranteed to be present in pretty much every bash, however it’s as limited as environment variables, it requires you to write a good bit of parsing code, and it has some surprising behavior regarding parsing order, option bundling, and error reporting. But it does allow you to have a `--help`, assuming you are willing to go to the trouble of coding it (and accounting for the fact that `getopts` only handles `-h` not `--help`)
- `getopt` is about as simple as `getopts` and it gives you `--long-options`, but it’s implemented differently literally everywhere - busybox, GNU/Linux, MacOS, BSD, they all have their own `getopt` that behaves slightly differently. Your code may work great in one system and fail in mysterious ways on another system.

Further, regardless of which option you choose, there tends to be a certain class of problems that crop up:

- The code that generates your `--help` message is often located just far enough away from your option parsing code that your `--help` message can quickly and easily fall out of sync with your actual arguments unless you are very disciplined
- When you want to validate input arguments, you have to define some early-exit behavior in your code, and then you have to decide whether to continue parsing and collect more errors, or bail immediately, and do you print the help or not, and how do you link all that code together
- When you’re writing your parser, you often wind up with such a large `case` statement that it becomes difficult to keep the context of the parser in your head when you’re adding a new flag. It’s surprisingly common to wind up duplicating flags or adding a new flag for something that could be extended or refined.
- You might need to accept an argument that defines a `key=value` pair, now you have to parse that pair, break it apart, and store them separately (depending on your use case)
- You might need to accept multiple values for a given argument and build a list. How do you do this? Do you accept a string with space separated items? Do you accept the same argument over and over? How do you store this in your script’s internal state?

I feel the same way about bash that I do about C: it’s basically the perfect language for its domain (in bash’s case, gluing tasks together on a unix system). But this particular task in bash has always managed to really piss me off, because frankly, we can do so much better than what we are doing right now. This is one of the big reasons why people will abandon bash for the kinds of glue tasks it’s so good at, and move to something like `golang`, because there’s not a great interface between the user and the script for configuring options at runtime. Parsing arguments can be downright unfriendly to the user, in fact, let alone to the developer. It doesn’t have to be this way! There’s no reason why Java and Python and Ruby and Golang and literally every other language out there can have high quality argument parsing libraries but Bash can’t have them.

So I wrote cmdarg: A pure bash argument parsing library. This library solves all of the above problems you will suffer through in all of the above scenarios. Any time I need to add arguments to a bash script, I reach for this library, and I am always pleased with the result.

## Declarative Syntax for Clear Interfaces

Cmdarg wants to help you build a clear interface to your script without asking you to write any of the parsing code. You want to write your script - you don’t want to write argument parsing code. So cmdarg allows you to declare the interface, parse the arguments, and get on with using the values in your script. Cmdarg assumes your script will take 0 or more command line arguments, and each one of these arguments will have:

- A name
- A type, including whether or not the argument is required or optional
- A description of what this argument does
- An optional default value for the argument
- An optional validation function to validate whatever the user gives the library

Additionally, cmdarg asks you to provide it with some metadata about your script:

- The name of the script author
- The copyright notice for the script
- A short description of the script suitable for inclusion in the help message

Your interface into cmdarg is simple and compact, and avoids sprawling information about your interface all over. The code winds up looking like this:

    #!/bin/bash
    source /usr/lib/cmdarg.sh
    declare -a myarray
    declare -A mymap

    cmdarg_info "header" "Some script that needed argument parsing"
    cmdarg_info "author" "Some Poor Bastard <somepoorbastard@hell.com>"
    cmdarg_info "copyright" "(C) 2013"
    cmdarg 'R:' 'required-thing' 'Some thing I REALLY require'
    cmdarg 'r:' 'required-thing-with-default' 'Some thing I require' 'Some default'
    cmdarg 'o?' 'optional-thing' 'Some optional thing'
    cmdarg 'b' 'boolean-thing' 'Some boolean thing'
    cmdarg 'a?[]' 'myarray' 'Some array of stuff'
    cmdarg 'm?{}' 'mymap' 'Some map of keys and values'
    cmdarg_parse "$@"

… and your user gets a clear usage message describing the interface, and you didn’t have to write a single line of extra code to do it:

    test.sh (C) 2013 : Some Poor Bastard <somepoorbastard@hell.com>

    Some script that needed argument parsing

    Required Arguments:
        -R,--required-thing v : String. Some thing I REALLY require

    Optional Arguments:
        -r,--required-thing-with-default v : String. Some thing I require (Default "Some default")
        -o,--optional-thing v : String. Some optional thing
        -b,--boolean-thing : Boolean. Some boolean thing
        -a,--myarray v[, ...] : Array. Some array of stuff. Pass this argument multiple times for multiple values.
        -m,--mymap k=v{, ..} : Hash. Some map of keys and values. Pass this argument multiple times for multiple key/value pairs.

## Clear interface for development

As a developer, using the parsed flags is equally simple. All simple options (strings and integers) get parsed into a toplevel map called `cmdarg_cfg`:

    echo "${cmdarg_cfg[required-thing]}"
    # ...

    $ script.sh -R "A moose once bit my sister"
    A moose once bit my sister

Array and map values are populated fully without any additional effort from you, presuming you have previously declared those arrays and maps, as per the documentation:

    echo "array items: ${myarray[@]}"
    echo "key items: ${!mymap[@]}"
    echo "key values: ${mymap[@]}"
    # ...

    $ script.sh -m key1=value1 -m key2=value2 -a thing1 -a thing2
    array items: thing1 thing2
    key items: key2 key1
    key values: value2 value1

As a script author, sometimes you want to say “Dump the entire configuration set passed to me by the user, as parsed by the parser, so I can see if something got FUBAR’ed in the parser”. There’s a function for that:

    cmdarg_dump
    #....

    $ script.sh -R "A moose once bit my sister" -m key1=value1 -m key2=value2 -a thing1 -a thing2
    boolean-thing:4 =>
    myarray:1 =>
            1 => thing1
            2 => thing2
    required-thing-with-default:3 => Some default
    optional-thing:3 =>
    mymap:2 =>
            key2 => value2
            key1 => value1
    required-thing:3 => A moose once bit my sister

## Positional arguments

cmdarg automatically adds any string on the command line that isn’t positioned as the value of another argument, and doesn’t begin with `-`, as a positional argument. Consider our example above:

    echo ${!cmdarg_argv[@]}
    echo ${cmdarg_argv[@]}
    # ...

    $ script.sh -R "A moose once bit my sister" -m key1=value1 positional_arg_1 -m key2=value2 -a thing1 positional_arg_2 -a thing2
    0 1
    positional_arg_1 positional_arg_2

… and cmdarg_dump shows them as well:

    cmdarg_dump
    # ...
    $ script.sh -R "A moose once bit my sister" -m key1=value1 positional_arg_1 -m key2=value2 -a thing1 positional_arg_2 -a thing2
    boolean-thing:4 =>
    myarray:1 =>
            1 => thing1
            2 => thing2
    required-thing-with-default:3 => Some default
    optional-thing:3 =>
    mymap:2 =>
            key2 => value2
            key1 => value1
    required-thing:3 => A moose once bit my sister
    argv =>
            0 => positional_arg_1
            1 => positional_arg_2

## Stop processing with –

Sometimes you need to provide an argument value that begins with `-`. The library must not interpret this as an argument, but as a value. The respected way of doing this is with `--` on the command line. cmdarg supports this natively:

    $ script.sh -R "A moose once bit my sister" -m key1=value1 positional_arg_1 -m key2=value2 -a thing1 positional_arg_2 -a thing2 -- -a -m --some_thing
    boolean-thing:4 =>
    myarray:1 =>
            1 => thing1
            2 => thing2
    required-thing-with-default:3 => Some default
    optional-thing:3 =>
    mymap:2 =>
            key2 => value2
            key1 => value1
    required-thing:3 => A moose once bit my sister
    argv =>
            0 => positional_arg_1
            1 => positional_arg_2
            2 => -a
            3 => -m
            4 => --some_thing

## There’s more. Go read the README

I’m not going to repeat the entirety of the README here, so go find out for yourself about:

- Using validator functions to validate user input on the arguments (including key names on map options)
- Using helper functions to control how arguments are described and how usage (`--help`) messages are constructed/printed
- Using custom error handlers to control what happens when the parser encounters an error in user input

## Why I still use this almost two decades later

A lot can change in 15 years. cmdarg is not the only game in town anymore. So why am I still using something I wrote 15 years ago? Well obviously I know this one, because I wrote it, so that’s an easy win. But also because, in 15 years, I have yet to find another library that meets all these requirements:

- Written purely in bash. Some contemporaries actually call some other language like python behind your back. Others rely on some complicated preprocessor that compiles your script down into a new script.
- Generate help messages that sufficiently document usage for the user, including all flags, optional vs required, array inputs, copyright, etc. Some contemporaries do this, but the formatting is kinda gross. Others simply omit it and still expect you to do it.
- Allows me to validate user input against custom rules. Some contemporaries provide this functionality but they have a built-in type system that is positively bonkers, as it is easy to go way too far with this kind of thing. Just let me give you a callback function that tells you if the input is valid or not, and get out of my way. And honestly speaking bash only has three types - strings, arrays, and maps. Pretending otherwise is unhealthy.
- Allows me to control error behavior. Sometimes I want parse errors or `--help` invocation to terminate the program, sometimes I don’t. You come across all kinds of wacky use cases in scripting. Some contemporaries meet almost all other requirements and are syntax-same to other known libraries, but they don’t allow for enough control over the parser’s behavior.
- Has a simple and clear interface. Quite frankly I think that a lot of other contemporaries simply have ugly interfaces. This is a purely personal aesthetic choice, and if we don’t share aesthetics, that’s fine. But I feel that cmdarg is downright elegant in not only what it does, but how it does it. Some tools have a delightful interface - I think the `bash_opts` single line spec to its `opts` function is really cool - but it doesn’t have enough other features.

## What frustrates me about this tool

This tool isn’t perfect and there are some things I continually think I should change. But, like how Plan 9 faded to antiquity while UNIX and Linux dominated the landscape, this tool is “good enough” that I just don’t see the need for some of these things.

The biggest thing I keep thinking I should add is sub-parsers. This is one thing that parsers in other languages like golang or python have a serious leg-up on us. Let’s say you’re writing a tool like this:

    $ sometool --help

    Usage: sometool [subcommand] <options>

    subcommands:
        file
    	socket
    	exec

    $ sometoool file --help

    Usage: sometool file <options>

    Options:
        --filename
    	--output

    $ sometoool socket --help

    Usage: sometool socket <options>

    Options:
        --host
    	--port

… cmdarg falls over here. Everything about cmdarg lives in the global namespace of the current script, so subparsers are difficult to implement. However, the reason I haven’t really worried about this TOO much, was summarized in a comment on a recently closed issue that I left open for 12 years:

> The right answer to your problem is to remember that you’re writing bash, not python, and you shouldn’t be creating submodules that live in the same space, you should be writing separate scripts. Entrypoint script A calls script B and presents it through the CLI as if it were some kind of submodule. Then script B does its own argument parsing with what script A passes it.

… and I’ll probably wind up implementing a nice little handler for doing that at some point in the near future. Something like

    cmdarg_submodule 'submodule_name' "/path/to/script/that/gets/submodule/arguments.sh"

… and that will probably be good enough to justify improving on this old tool in the toolbox.

Recent Posts
2026-05-27
News-2026-Week-2
2026-05-15
Devlog Entry
2026-02-23
Why Firefighting
2026-02-09
News 2026 Week 5