Python Argparse

Photo by Chris Ried on Unsplash

Photo by Chris Ried on Unsplash

Introduction

Python’s argparse module makes parsing command line arguments a breeze. The argparse library allows:

  • Positional arguments

  • Customization of prefix characters

  • Variable numbers of parameters for a single option

  • Subcommands

I’ll be doing this in Ubuntu 18.04, but this should apply to all Linux distributions. Since Python is portable across Windows and Mac it should apply to those as well. Your mileage may vary, caveat emptor, etc.

The Command Line

terminal-blank.jpg

My terminal looks like the image to the right. This is a sample of what the command line looks like in Ubuntu. Most Linux distributions ship with the Bourne Again or bash shell. At this time I’m using zsh, but we will use bash since it’s pretty much the de facto standard at this point. Other shells include the Bourne shell, tcsh shell, korn shell, and the Fish shell.

A few pointers about the shell:

  • Entering cd at the prompt will take you to your home directory

  • I never use the rm command when I can avoid it! Use trash instead

  • Entering sudo <command> at the prompt will execute the listed command as the root user if you are a member of the sudo group

  • cd - will take you back to the previous directory. This is useful if you accidentally enter a command such as cd by accident and can’t remember where you were

  • pwd, cp, mv, mkdir, ls, and mount are all examples of things that can be done at the command line

Consider the following command:

cp -i --preserve=mode,ownership source destination

There is a lot going on in that single line, so let’s break it down:

  • The -i is an option that says to use interactive mode, meaning we get prompted if we are sure we want to remove the destination file if it already exists

  • The --preserve=mode,ownership provide additional parameters specifying we’d like to keep both the mode and the owner of the file but NOT, e.g., the time stamp.

  • The source and destination are arguments that specify the origin and destination of the file we’d like to copy

The Basics

Let’s dig into the details of using argparse. Consider the following Python code (we’ll call this particular file mycp.py):

import os, sys

if len(sys.argv) < 3:
print('You must specify at least two arguments\n')

This tells Python to:

  • import the os and sys modules

  • verify that sys.argv is not less than 3

    • if it is less than 3, print an error

Now try to run the program with no command line arguments like so:

python mycp.py

And you will get this from the Python interpreter:

You must specify at least one argument

Traceback (most recent call last):
File "mycp.py", line 7, in <module>
input_path = sys.argv[1]
IndexError: list index out of range

This simply says we have provided no arguments. Run it again with arguments:

python mycp.py infile.txt outfile.txt

and we get no output, which, in this case, is what we want.

Now let’s modify the code a little:

import os, sys
import argparse

if len(sys.argv) < 3:
print('You must specify at least two arguments\n')

input_path = sys.argv[1]
if not os.path.isfile(input_path):
print("The specified file does not exist")

output_path = sys.argv[2]

We added:

  • The import argparse statement

  • Assigned the variable input_path to to take on the value of sys.argv[1]

    • Review Python lists if you need to understand what sys.argv[1] means

  • With the os.path module, test to see if input_path is a file

    • If it is not, print an error

  • Assigned the variable output_path to take

Run this code and we get:

The specified file does not exist

This is because we have yet to create a file for the input_path variable. To get around this error type:

touch infile.txt

Then run the program again. Again we should get no output.

Getting Fancy

We are going to re-implement a slightly simpler form of cp than exists in the shell. Modify your Python script as follows:

import os, sys
import argparse
import shutil

if len(sys.argv) < 3:
print('You must specify at least two arguments\n')

input_path = sys.argv[1]
if not os.path.isfile(input_path):
print("The specified file does not exist")

output_path = sys.argv[2]

shutil.copyfile(input_path, output_path)

What we have done here is:

  • import the shutil module

  • used it to copy our input_path to our output_path

Run your Python script again. If you were successful you see no output again. However, this time your script did something! At the prompt type:

ls

This should output the something like the following:

infile.txt mycp.py outfile.txt

We already know why infile.txt is there, but the outfile.txt is new! Unfortunately infile.txt is empty — we should put something in there! My favorite editor for doing so is gvim, but editors include nano, vi, vim, emacs, sublime, and more — there are even full on IDEs like eclipse or vscode. If you’d like to use gvim on Ubuntu you install it with:

sudo apt install vim-gtk

Tutorials on vim can be found online. If you are using anything other than vim, refer to your individual editor’s documentation.

Edit infile.txt with something like the following (in vi this is done by pushing ‘i’ to enter Insert mode):

The quick brown fox jumped over the lazy dog.

If you are using vim you save and exit by entering <ESC> (to exit Insert mode) followed by ‘:wq’ in the vim window.

Now rerun your program. If you have completed all the tasks correctly to this point enter: (at the prompt):

cat outfile.txt

and this should print:

The quick brown fox jumped over the lazy dog.

Huzzah! Incidentally, cat stands for concatenate.

Getting Freaky

Now let’s do the same thing using a system call to cp. Add the following line to your Python script:

os.system("cp -i --preserve=mode,ownership " + input_path + " " + output_path)

This tells Python to make a system call to the shell as opposed to using the shutil module. We should see the following when we run the script:

cp: overwrite 'outfile.txt'?

Go ahead and enter ‘y’ at the prompt. If you cat outfile.txt you should see the same thing you did before.

Getting Downright Elaborate

PRO TIP: You cannot name your file “argparse.py” — otherwise Python will use it in place of the “real” argparse module. See stackoverflow.

Create a new script called ‘cp-via-argparse.py’ via touch, vim, or your favorite editor as follows:

import os, sys
import argparse

#Create the parser
a_parser = argparse.ArgumentParser(description='cp implementation')

#Add the arguments
a_parser.add_argument('Infile', metavar='infile', type=str,
help='The file to be copied')
a_parser.add_argument('Outfile', metavar='outfile', type=str, nargs='?',
help='The destination file', default='outfile.txt')

#Execute the parse_args() method
args = a_parser.parse_args()

if not os.path.isfile(args.Infile):
print("The path specified does not exist")
sys.exit()

#Make a system call to the shell
os.system("cp -i --preserve=mode,ownership " + args.Infile + " " + args.Outfile)

What this script is doing is:

  • importing the modules we need

  • creating an ArgumentParser object

  • adding two arguments

  • assigning args to argparse.Namespace object then executing the parse_args method

  • verifying that args.Infile is actually a file

    • if it’s not, print an error and exit the Python interpreter

  • makes a system call as before but using the Infile and Outfile arguments as shown

Now if you run this code without arguments you will get somewhat different output:

python3 cp-via-argparse.py
usage: cp-via-argparse.py [-h] infile outfile
cp-via-argparse.py: error: the following arguments are required: infile

What’s happening is that Python is expecting at least one positional argument. When we give it one we get:

python3 cp-via-argparse.py infile.txt
cp: overwrite 'outfile.txt'?

Go ahead and type y and the program should exit.

Another benefit to the argparse module is contextual help. Enter:

python3 cp-via-argparse.py -h

at a prompt and you’ll get a helpful usage statement:

usage: cp-via-argparse.py [-h] infile [outfile]

cp implementation

positional arguments:
infile The file to be copied
outfile The destination file

optional arguments:
-h, --help show this help message and exit

A Note On Options

Optional arguments are not mandatory, and when they are used they can modify the behavior of the command at runtime. In the cp example, an optional argument is, for example, the -r flag, which makes the command copy directories recursively.

Syntactically, the difference between positional and optional arguments is that optional arguments start with - or --, while positional arguments don’t.

Let’s modify our cp-via-argparse.py script one more time:

#Add the arguments
a_parser.add_argument('Infile', metavar='infile', type=str,
help='The file to be copied')
a_parser.add_argument('Outfile', metavar='outfile', type=str, nargs='?',
help='The destination file', default='outfile.txt')

#Optional arguments
a_parser.add_argument('-i', dest='overwrite_prompt', action='store_true', help='Prompt on existing file')
a_parser.add_argument('--preserve', dest='preserve', type=str, nargs='?', help='Preserve (see man page for cp)')

#Execute the parse_args() method
test = 'infile.txt outfile.txt -i --preserve=all'
args = a_parser.parse_args(test.split())

if not os.path.isfile(args.Infile):
print("The path specified does not exist")
sys.exit()

options = ''
if args.overwrite_prompt:
options = '-i '


#System call to the shell
os.system("cp " + options + "--preserve=" + args.preserve + " " + args.Infile + " " + args.Outfile)

We’ve added:

  • two optional arguments, ‘-i’ and ‘--preserve’

  • introduced the split method and used it to divide our arguments

  • did some fancy string processing with args.overwtite_prompt

  • modified our system call as shown

And that’s it! You successfully used the Python argparse module!

Gory Details

I borrowed heavily from this great article and argparse is covered to a greater degree there.

The argparse library was released as part of the standard library with Python 3.2 as part of the Python Enhancement Proposal 389. It was a replacement for the getopt and optparse modules.

https://docs.python.org/3.8/library/argparse.html

https://docs.python.org/3/howto/argparse.html

https://pymotw.com/2/argparse/

http://zetcode.com/python/argparse/

Previous
Previous

State Machines

Next
Next

Creating An Untouchable Space