Python Argparse
Photo by Chris Ried on Unsplash
Introduction
Python’s argparse module makes parsing command line arguments a breeze. The argparse library allows:
Positional arguments
Customization of prefix characters
Variable numbers of parameters for a single option
Subcommands
I’ll be doing this in Ubuntu 18.04, but this should apply to all Linux distributions. Since Python is portable across Windows and Mac it should apply to those as well. Your mileage may vary, caveat emptor, etc.
The Command Line
My terminal looks like the image to the right. This is a sample of what the command line looks like in Ubuntu. Most Linux distributions ship with the Bourne Again or bash shell. At this time I’m using zsh, but we will use bash since it’s pretty much the de facto standard at this point. Other shells include the Bourne shell, tcsh shell, korn shell, and the Fish shell.
A few pointers about the shell:
Entering cd at the prompt will take you to your home directory
I never use the rm command when I can avoid it! Use trash instead
Entering sudo <command> at the prompt will execute the listed command as the root user if you are a member of the sudo group
cd - will take you back to the previous directory. This is useful if you accidentally enter a command such as cd by accident and can’t remember where you were
pwd, cp, mv, mkdir, ls, and mount are all examples of things that can be done at the command line
Consider the following command:
cp -i --preserve=mode,ownership source destination
There is a lot going on in that single line, so let’s break it down:
The -i is an option that says to use interactive mode, meaning we get prompted if we are sure we want to remove the destination file if it already exists
The --preserve=mode,ownership provide additional parameters specifying we’d like to keep both the mode and the owner of the file but NOT, e.g., the time stamp.
The source and destination are arguments that specify the origin and destination of the file we’d like to copy
The Basics
Let’s dig into the details of using argparse. Consider the following Python code (we’ll call this particular file mycp.py):
import os, sys
if len(sys.argv) < 3:
print('You must specify at least two arguments\n')
This tells Python to:
verify that sys.argv is not less than 3
if it is less than 3, print an error
Now try to run the program with no command line arguments like so:
python mycp.py
And you will get this from the Python interpreter:
You must specify at least one argument
Traceback (most recent call last):
File "mycp.py", line 7, in <module>
input_path = sys.argv[1]
IndexError: list index out of range
This simply says we have provided no arguments. Run it again with arguments:
python mycp.py infile.txt outfile.txt
and we get no output, which, in this case, is what we want.
Now let’s modify the code a little:
import os, sys
import argparse
if len(sys.argv) < 3:
print('You must specify at least two arguments\n')
input_path = sys.argv[1]
if not os.path.isfile(input_path):
print("The specified file does not exist")output_path = sys.argv[2]
We added:
The import argparse statement
Assigned the variable input_path to to take on the value of sys.argv[1]
Review Python lists if you need to understand what sys.argv[1] means
With the os.path module, test to see if input_path is a file
If it is not, print an error
Assigned the variable output_path to take
Run this code and we get:
The specified file does not exist
This is because we have yet to create a file for the input_path variable. To get around this error type:
touch infile.txt
Then run the program again. Again we should get no output.
Getting Fancy
We are going to re-implement a slightly simpler form of cp than exists in the shell. Modify your Python script as follows:
import os, sys
import argparse
import shutil
if len(sys.argv) < 3:
print('You must specify at least two arguments\n')
input_path = sys.argv[1]
if not os.path.isfile(input_path):
print("The specified file does not exist")
output_path = sys.argv[2]
shutil.copyfile(input_path, output_path)
What we have done here is:
import the shutil module
used it to copy our input_path to our output_path
Run your Python script again. If you were successful you see no output again. However, this time your script did something! At the prompt type:
ls
This should output the something like the following:
infile.txt mycp.py outfile.txt
We already know why infile.txt is there, but the outfile.txt is new! Unfortunately infile.txt is empty — we should put something in there! My favorite editor for doing so is gvim, but editors include nano, vi, vim, emacs, sublime, and more — there are even full on IDEs like eclipse or vscode. If you’d like to use gvim on Ubuntu you install it with:
sudo apt install vim-gtk
Tutorials on vim can be found online. If you are using anything other than vim, refer to your individual editor’s documentation.
Edit infile.txt with something like the following (in vi this is done by pushing ‘i’ to enter Insert mode):
The quick brown fox jumped over the lazy dog.
If you are using vim you save and exit by entering <ESC> (to exit Insert mode) followed by ‘:wq’ in the vim window.
Now rerun your program. If you have completed all the tasks correctly to this point enter: (at the prompt):
cat outfile.txt
and this should print:
The quick brown fox jumped over the lazy dog.
Huzzah! Incidentally, cat stands for concatenate.
Getting Freaky
Now let’s do the same thing using a system call to cp. Add the following line to your Python script:
os.system("cp -i --preserve=mode,ownership " + input_path + " " + output_path)
This tells Python to make a system call to the shell as opposed to using the shutil module. We should see the following when we run the script:
cp: overwrite 'outfile.txt'?
Go ahead and enter ‘y’ at the prompt. If you cat outfile.txt you should see the same thing you did before.
Getting Downright Elaborate
PRO TIP: You cannot name your file “argparse.py” — otherwise Python will use it in place of the “real” argparse module. See stackoverflow.
Create a new script called ‘cp-via-argparse.py’ via touch, vim, or your favorite editor as follows:
import os, sys
import argparse
#Create the parser
a_parser = argparse.ArgumentParser(description='cp implementation')
#Add the arguments
a_parser.add_argument('Infile', metavar='infile', type=str,
help='The file to be copied')
a_parser.add_argument('Outfile', metavar='outfile', type=str, nargs='?',
help='The destination file', default='outfile.txt')
#Execute the parse_args() method
args = a_parser.parse_args()
if not os.path.isfile(args.Infile):
print("The path specified does not exist")
sys.exit()#Make a system call to the shell
os.system("cp -i --preserve=mode,ownership " + args.Infile + " " + args.Outfile)
What this script is doing is:
importing the modules we need
creating an ArgumentParser object
adding two arguments
assigning args to argparse.Namespace object then executing the parse_args method
verifying that args.Infile is actually a file
if it’s not, print an error and exit the Python interpreter
makes a system call as before but using the Infile and Outfile arguments as shown
Now if you run this code without arguments you will get somewhat different output:
python3 cp-via-argparse.py
usage: cp-via-argparse.py [-h] infile outfile
cp-via-argparse.py: error: the following arguments are required: infile
What’s happening is that Python is expecting at least one positional argument. When we give it one we get:
python3 cp-via-argparse.py infile.txt
cp: overwrite 'outfile.txt'?
Go ahead and type y and the program should exit.
Another benefit to the argparse module is contextual help. Enter:
python3 cp-via-argparse.py -h
at a prompt and you’ll get a helpful usage statement:
usage: cp-via-argparse.py [-h] infile [outfile]
cp implementation
positional arguments:
infile The file to be copied
outfile The destination file
optional arguments:
-h, --help show this help message and exit
A Note On Options
Optional arguments are not mandatory, and when they are used they can modify the behavior of the command at runtime. In the cp example, an optional argument is, for example, the -r flag, which makes the command copy directories recursively.
Syntactically, the difference between positional and optional arguments is that optional arguments start with - or --, while positional arguments don’t.
Let’s modify our cp-via-argparse.py script one more time:
#Add the arguments
a_parser.add_argument('Infile', metavar='infile', type=str,
help='The file to be copied')
a_parser.add_argument('Outfile', metavar='outfile', type=str, nargs='?',
help='The destination file', default='outfile.txt')
#Optional arguments
a_parser.add_argument('-i', dest='overwrite_prompt', action='store_true', help='Prompt on existing file')
a_parser.add_argument('--preserve', dest='preserve', type=str, nargs='?', help='Preserve (see man page for cp)')
#Execute the parse_args() method
test = 'infile.txt outfile.txt -i --preserve=all'
args = a_parser.parse_args(test.split())
if not os.path.isfile(args.Infile):
print("The path specified does not exist")
sys.exit()
options = ''
if args.overwrite_prompt:
options = '-i '
#System call to the shell
os.system("cp " + options + "--preserve=" + args.preserve + " " + args.Infile + " " + args.Outfile)
We’ve added:
two optional arguments, ‘-i’ and ‘--preserve’
introduced the split method and used it to divide our arguments
did some fancy string processing with args.overwtite_prompt
modified our system call as shown
And that’s it! You successfully used the Python argparse module!
Gory Details
I borrowed heavily from this great article and argparse is covered to a greater degree there.
The argparse library was released as part of the standard library with Python 3.2 as part of the Python Enhancement Proposal 389. It was a replacement for the getopt and optparse modules.
https://docs.python.org/3.8/library/argparse.html
https://docs.python.org/3/howto/argparse.html
