Sharpen your Python debugging skills

Debugging is a fundamental skill for programmers, it’s a craft and we need to practice to be better at it. Here, I summarized the useful tricks and tools for debugging your Python code.

#1 View The Source Code Of A Function Or Object

Know the details of implementation is critical in debugging. When you are debugging a Python code, you may want to know the source code of a module, class, method, function, traceback, frame, or code object. The inspect module will help you:

import inspect
def add(x, y):
    return x + y

class Dog:
    kind = 'dog'        
    def __init__(self, name):
        self.name = name
    def eat(food):
        print("eating ..." + food)
        
print(inspect.getsource(add))            
print(inspect.getsource(Dog))
print(inspect.getsource(Dog.eat))

#2 Print Log Into File

The log is the most important clue for us to understanding the runtime of code. Printing out log is a useful way to follow the program’s execution flow.

However, print in Python only outputs to a terminal. It’s far from ideal for complex troubleshooting, especially for server-side applications.

The print function in Python 3 is much more powerful because it can take more arguments, and specifying some arguments can output the contents of print to a log file.

with open('test.log', mode='w') as f:
    print('hello, python', file=f, flush=True)

Another useful and flexible module is logging, we could set the output filename, log format, etc..

import logging
logging.basicConfig(filename='app.log', filemode='w', 
  format='%(name)s - %(levelname)s - %(message)s')
a = 5
b = 0
try:
  c = a / b
except Exception as e:
  logging.exception("Exception occurred")

For more usage please refer to logging — Logging facility for Python — Python 3.9.0 documentation.

#3 Track The Execution Time

For performance issues, we need to calculate the run time of a function. You might achieve it like this:

import timestart = time.time()
# run the function
end = time.time()
print(end-start)

There’s a built-in module called timeit, we could use it to track the time with only one line of code:

import time
import timeit

def run_sleep(second):
    print(second)
    time.sleep(second)

print(timeit.timeit(lambda :run_sleep(2), number=1))

#4 Interactive Debugging

REPL supports an incremental, interactive development style, and interactive debugging is a more efficient way. The ideal workflow is the repeated process of edit-run.

To invoke a program with an interactive session, we need to add the -i option:

python -i demo.py

To make your life easier, importlib.reload(module) is a suitable way to avoid restart the interactive session.

Let’s take an example, suppose we are run a function called demo, to fix an issue we need to make some changes for this function, then reload(module) will load the modified version of demo without restart our session:

>>> from importlib import reload
>>> import demo from module
>>> demo()
"The result of demo..."# Make some changes to "demo"
>>> demo()
"The result of demo..."  # The Outdated result
>>> reload(module)  # Reload "module" after changes made to "demo"
>>> demo()
"This will be the new result..."
>>> # repeat the circle of fix-and-retry

This will save much time for debugging.

#5 Use Pdb

Python has a module called pdb to support interactive source code debugger. It supports setting breakpoints, single stepping at the line level. In addition, pdb print inspection of stack frames, values of variables, inspect source code, etc.

To start a program with a debugger, add -m pdb option for invoking scripts:

user@coderscat:~/snippets$ python3.7 -m pdb demo.py 
> /home/user/snippets/demo.py(1)<module>()
-> import inspect
(Pdb) l
  1  -> import inspect
  2  
  3     def gcd(a,b):
  4         if(b==0):
  5             return a
  6         else:
  7             return gcd(b,a%b)
  8  
  9  
 10     print(gcd(34, 34))
[EOF]
(Pdb) b demo.py:3
Breakpoint 1 at /home/user/snippets/demo.py:3
(Pdb) c
> /home/user/snippets/demo.py(3)<module>()
-> def gcd(a,b):
(Pdb) step
> /home/user/snippets/demo.py(10)<module>()
-> print(gcd(34, 34))
(Pdb)

Another typical usage to break into the debugger from a running program is to insert:

import pdb; pdb.set_trace()

Here are some useful pdb command

list(l): show you the code line the Python interpreter is currently on
step(s): continue the execution line by line, step in function
next(n) : continue the next line of code
break(b) : setup new breakpoint on current line
continue(c) : continue execution until next breakpoint

In most cases, we don’t need to follow every step in execution, pdb is the ultimate tool for debugging Python program.

Conclusion

When you debug Python code, have a try on these tricks and tools. To improve your skills in debugging, practice makes perfect!