Skip to content

Usage

Debug🔗

Somehow in the 27 years (and counting) of active development of python, no one thought to add a simple and readable way to print stuff during development. (If you know why this is, I'd love to hear).

The wait is over:

from devtools import debug

v1 = {
    'foo': {1: 'nested', 2: 'dict'},
    'bar': ['apple', 'banana', 'carrot', 'grapefruit'],
}

debug(v1, sum(range(5)))

example.py:8 <module>
    v1: {
        'foo': {
            1: 'nested',
            2: 'dict',
        },
        'bar': [
            'apple',
            'banana',
            'carrot',
            'grapefruit',
        ],
    } (dict) len=2
    sum(range(5)): 10 (int)

debug is like print after a good night's sleep and lots of coffee:

  • each output is prefixed with the file, line number and function where debug was called
  • the variable name or expression being printed is shown
  • each argument is printed "pretty" on a new line, see prettier print
  • if pygments is installed the output is highlighted

A more complex example of debug shows more of what it can do.

from devtools import debug
import numpy as np

foo = {
    'foo': np.array(range(20)),
    'bar': [{'a': i, 'b': {j for j in range(1 + i * 2)}} for i in range(3)],
    'spam': (i for i in ['i', 'am', 'a', 'generator']),
}

debug(foo)

# kwargs can be used as keys for what you are printing
debug(
    long_string='long strings get wrapped ' * 10,
    new_line_string='wraps also on newline\n' * 3,
)

bar = {1: 2, 11: 12}
# debug can also show the output of expressions
debug(
    len(foo),
    bar[1],
    foo == bar
)

complex.py:10 <module>
    foo: {
        'foo': (
            array([ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, 13, 14,
            15, 16,
                   17, 18, 19])
        ),
        'bar': [
            {
                'a': 0,
                'b': {0},
            },
            {
                'a': 1,
                'b': {0, 1, 2},
            },
            {
                'a': 2,
                'b': {
                    0,
                    1,
                    2,
                    3,
                    4,
                },
            },
        ],
        'spam': (
            'i',
            'am',
            'a',
            'generator',
        ),
    } (dict) len=3
complex.py:15 <module>
    long_string: (
        'long strings get wrapped long strings get wrapped long strings get wr'
        'apped long strings get wrapped long strings get wrapped long strings '
        'get wrapped long strings get wrapped long strings get wrapped long st'
        'rings get wrapped long strings get wrapped '
    ) (str) len=250
    new_line_string: (
        'wraps also on newline\n'
        'wraps also on newline\n'
        'wraps also on newline\n'
    ) (str) len=66
complex.py:23 <module>
    len(foo): 3 (int)
    bar[1]: 2 (int)
    foo == bar: False (bool)

Other debug tools🔗

The debug namespace includes a number of other useful functions:

  • debug.format() same as calling debug() but returns a DebugOutput rather than printing the output
  • debug.timer() returns an instance of devtool's Timer class suitable for timing code execution
  • debug.breakpoint() introduces a breakpoint using pdb
import random

from devtools import debug

# debug.format() behaves the same as debug() except it
# returns an object to inspect or print
r = debug.format(x=123, y=321)
print(r)
print(repr(r))

values = list(range(int(1e5)))
# timer can be used as a context manager or directly
with debug.timer('shuffle values'):
    random.shuffle(values)

t2 = debug.timer('sort values').start()
sorted(values)
t2.capture()

# if used repeatedly a summary is available
t3 = debug.timer()
for i in [1e4, 1e6, 1e7]:
    with t3('sum {}'.format(i), verbose=False):
        sum(range(int(i)))

t3.summary(verbose=True)

# debug.breakpoint()
# would drop to a prompt:
# > /python-devtools/docs/examples/more_debug.py(28)<module>()->None
# -> debug.breakpoint()
# (Pdb)

other.py:7 <module>
    x: 123 (int)
    y: 321 (int)
<DebugOutput other.py:7 <module> arguments: x: 123 (int) y: 321 (int)>
shuffle values: 0.074s elapsed
sort values: 0.024s elapsed
    sum 10000.0: 0.000s elapsed
    sum 1000000.0: 0.019s elapsed
    sum 10000000.0: 0.188s elapsed
3 times: mean=0.069s stdev=0.104s min=0.000s max=0.188s

Prettier print🔗

Python comes with pretty print, problem is quite often it's not that pretty, it also doesn't cope well with non standard python objects (think numpy arrays or django querysets) which have their own pretty print functionality.

To get round this devtools comes with prettier print, my take on pretty printing. You can see it in use above in debug(), but it can also be used directly:

from devtools import PrettyFormat, pprint, pformat
import numpy as np

v = {
    'foo': {'whatever': [3, 2, 1]},
    'sentence': 'hello\nworld',
    'generator': (i * 2 for i in [1, 2, 3]),
    'matrix': np.matrix([[1, 2, 3, 4],
                         [50, 60, 70, 80],
                         [900, 1000, 1100, 1200],
                         [13000, 14000, 15000, 16000]])
}

# pretty print of v
pprint(v)

# as above without colours, the generator will also be empty as
# it's already been evaluated
s = pformat(v, highlight=False)
print(s)

pp = PrettyFormat(
    indent_step=2,  # default: 4
    indent_char='.',  # default: space
    repr_strings=True,  # default: False
    # default: 10 (if line is below this length it'll be shown on one line)
    simple_cutoff=2,
    width=80,  # default: 120
    # default: True (whether to evaluate generators
    yield_from_generators=False,
)

print(pp(v, highlight=True))

{
    'foo': {
        'whatever': [3, 2, 1],
    },
    'sentence': (
        'hello\n'
        'world'
    ),
    'generator': (
        2,
        4,
        6,
    ),
    'matrix': (
        matrix([[    1,     2,     3,     4],
                [   50,    60,    70,    80],
                [  900,  1000,  1100,  1200],
                [13000, 14000, 15000, 16000]])
    ),
}
{
    'foo': {
        'whatever': [3, 2, 1],
    },
    'sentence': (
        'hello\n'
        'world'
    ),
    'generator': (
    ),
    'matrix': (
        matrix([[    1,     2,     3,     4],
                [   50,    60,    70,    80],
                [  900,  1000,  1100,  1200],
                [13000, 14000, 15000, 16000]])
    ),
}
{
..'foo': {
....'whatever': [
......3,
......2,
......1,
....],
..},
..'sentence': 'hello\nworld',
..'generator': <generator object <genexpr> at 0x7f3cbf386f50>,
..'matrix': (
....matrix([[    1,     2,     3,     4],
....        [   50,    60,    70,    80],
....        [  900,  1000,  1100,  1200],
....        [13000, 14000, 15000, 16000]])
..),
}

For more details on prettier printing, see prettier.py.

ANSI terminal colours🔗

from devtools import sprint, sformat

sprint('this is red', sprint.red)

sprint('this is bold underlined blue on a green background. yuck',
       sprint.blue, sprint.bg_yellow, sprint.bold, sprint.underline)

v = sformat('i am dim', sprint.dim)
print(repr(v))
# > '\x1b[2mi am dim\x1b[0m'

For more details on ansi colours, see ansi.py.

Usage without import🔗

We all know the annoyance of running code only to discover a missing import, this can be particularly frustrating when the function you're using isn't used except during development.

You can setup your environment to make debug available at all times by editing sitecustomize.py, with ubuntu and python3.6 this file can be found at /usr/lib/python3.6/sitecustomize.py but you might need to look elsewhere depending on your OS/python version.

Add the following to sitecustomize.py

...

try:
    from devtools import debug
except ImportError:
    pass
else:
    __builtins__['debug'] = debug

The ImportError exception is important since you'll want python to run fine even if devtools isn't installed.

This approach has another advantage: if you forget to remove debug(...) calls from your code, CI (which won't have devtools installed) should fail both on execution and linting, meaning you don't end up with extraneous debug calls in production code.