Skip to content

Usage

Debug🔗

Somehow in the 27 years (and counting) of active development of python, no one thought to add a simple and readable way to print stuff during development. (If you know why this is, I'd love to hear).

The wait is over:

from devtools import debug

v1 = {
    'foo': {1: 'nested', 2: 'dict'},
    'bar': ['apple', 'banana', 'carrot', 'grapefruit'],
}

debug(v1, sum(range(5)))

docs/examples/example.py:8 <module>
    v1: {
        'foo': {
            1: 'nested',
            2: 'dict',
        },
        'bar': [
            'apple',
            'banana',
            'carrot',
            'grapefruit',
        ],
    } (dict) len=2
    sum(range(5)): 10 (int)

debug is like print after a good night's sleep and lots of coffee:

  • each output is prefixed with the file, line number and function where debug was called
  • the variable name or expression being printed is shown
  • each argument is printed "pretty" on a new line, see prettier print
  • if pygments is installed the output is highlighted

A more complex example of debug shows more of what it can do.

import numpy as np

from devtools import debug

foo = {
    'foo': np.array(range(20)),
    'bar': [{'a': i, 'b': {j for j in range(1 + i * 2)}} for i in range(3)],
    'spam': (i for i in ['i', 'am', 'a', 'generator']),
}

debug(foo)

# kwargs can be used as keys for what you are printing
debug(
    long_string='long strings get wrapped ' * 10,
    new_line_string='wraps also on newline\n' * 3,
)

bar = {1: 2, 11: 12}
# debug can also show the output of expressions
debug(
    len(foo),
    bar[1],
    foo == bar
)

docs/examples/complex.py:11 <module>
    foo: {
        'foo': (
            array([ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, 13, 14,
            15, 16,
                   17, 18, 19])
        ),
        'bar': [
            {
                'a': 0,
                'b': {0},
            },
            {
                'a': 1,
                'b': {0, 1, 2},
            },
            {
                'a': 2,
                'b': {
                    0,
                    1,
                    2,
                    3,
                    4,
                },
            },
        ],
        'spam': (
            'i',
            'am',
            'a',
            'generator',
        ),
    } (dict) len=3
docs/examples/complex.py:14 <module>
    long_string: (
        'long strings get wrapped long strings get wrapped long strings get wr'
        'apped long strings get wrapped long strings get wrapped long strings '
        'get wrapped long strings get wrapped long strings get wrapped long st'
        'rings get wrapped long strings get wrapped '
    ) (str) len=250
    new_line_string: (
        'wraps also on newline\n'
        'wraps also on newline\n'
        'wraps also on newline\n'
    ) (str) len=66
docs/examples/complex.py:21 <module>
    len(foo): 3 (int)
    bar[1]: 2 (int)
    foo == bar: False (bool)

Returning the arguments🔗

debug will return the arguments passed to it meaning you can insert debug(...) into code.

The returned arguments work as follows:

  • if one non-keyword argument is passed to debug(), it is returned as-is
  • if multiple arguments are passed to debug(), they are returned as a tuple
  • if keyword arguments are passed to debug(), the kwargs dictionary is added to the returned tuple
from devtools import debug

assert debug('foo') == 'foo'
assert debug('foo', 'bar') == ('foo', 'bar')
assert debug('foo', 'bar', spam=123) == ('foo', 'bar', {'spam': 123})
assert debug(spam=123) == ({'spam': 123},)

docs/examples/return_args.py:3 <module>
    'foo' (str) len=3
docs/examples/return_args.py:4 <module>
    'foo' (str) len=3
    'bar' (str) len=3
docs/examples/return_args.py:5 <module>
    'foo' (str) len=3
    'bar' (str) len=3
    spam: 123 (int)
docs/examples/return_args.py:6 <module>
    spam: 123 (int)

Other debug tools🔗

The debug namespace includes a number of other useful functions:

  • debug.format() same as calling debug() but returns a DebugOutput rather than printing the output
  • debug.timer() returns an instance of devtool's Timer class suitable for timing code execution
  • debug.breakpoint() introduces a breakpoint using pdb
import random

from devtools import debug

# debug.format() behaves the same as debug() except it
# returns an object to inspect or print
r = debug.format(x=123, y=321)
print(r)
print(repr(r))

values = list(range(int(1e5)))
# timer can be used as a context manager or directly
with debug.timer('shuffle values'):
    random.shuffle(values)

t2 = debug.timer('sort values').start()
sorted(values)
t2.capture()

# if used repeatedly a summary is available
t3 = debug.timer()
for i in [1e4, 1e6, 1e7]:
    with t3(f'sum {i}', verbose=False):
        sum(range(int(i)))

t3.summary(verbose=True)

# debug.breakpoint()
# would drop to a prompt:
# > /python-devtools/docs/examples/more_debug.py(28)<module>()->None
# -> debug.breakpoint()
# (Pdb)

docs/examples/other.py:7 <module>
    x: 123 (int)
    y: 321 (int)
<DebugOutput docs/examples/other.py:7 <module> arguments: x: 123 (int) y: 321 (int)>
shuffle values: 0.057s elapsed
sort values: 0.023s elapsed
    sum 10000.0: 0.000s elapsed
    sum 1000000.0: 0.025s elapsed
    sum 10000000.0: 0.246s elapsed
3 times: mean=0.090s stdev=0.136s min=0.000s max=0.246s

Prettier print🔗

Python comes with pretty print, problem is quite often it's not that pretty, it also doesn't cope well with non standard python objects (think numpy arrays or django querysets) which have their own pretty print functionality.

To get round this devtools comes with prettier print, my take on pretty printing. You can see it in use above in debug(), but it can also be used directly:

import numpy as np

from devtools import PrettyFormat, pformat, pprint

v = {
    'foo': {'whatever': [3, 2, 1]},
    'sentence': 'hello\nworld',
    'generator': (i * 2 for i in [1, 2, 3]),
    'matrix': np.matrix([[1, 2, 3, 4],
                         [50, 60, 70, 80],
                         [900, 1000, 1100, 1200],
                         [13000, 14000, 15000, 16000]])
}

# pretty print of v
pprint(v)

# as above without colours, the generator will also be empty as
# it's already been evaluated
s = pformat(v, highlight=False)
print(s)

pp = PrettyFormat(
    indent_step=2,  # default: 4
    indent_char='.',  # default: space
    repr_strings=True,  # default: False
    # default: 10 (if line is below this length it'll be shown on one line)
    simple_cutoff=2,
    width=80,  # default: 120
    # default: True (whether to evaluate generators
    yield_from_generators=False,
)

print(pp(v, highlight=True))

{
    'foo': {
        'whatever': [3, 2, 1],
    },
    'sentence': (
        'hello\n'
        'world'
    ),
    'generator': (
        2,
        4,
        6,
    ),
    'matrix': (
        matrix([[    1,     2,     3,     4],
                [   50,    60,    70,    80],
                [  900,  1000,  1100,  1200],
                [13000, 14000, 15000, 16000]])
    ),
}
{
    'foo': {
        'whatever': [3, 2, 1],
    },
    'sentence': (
        'hello\n'
        'world'
    ),
    'generator': (
    ),
    'matrix': (
        matrix([[    1,     2,     3,     4],
                [   50,    60,    70,    80],
                [  900,  1000,  1100,  1200],
                [13000, 14000, 15000, 16000]])
    ),
}
{
..'foo': {
....'whatever': [
......3,
......2,
......1,
....],
..},
..'sentence': 'hello\nworld',
..'generator': <generator object <genexpr> at 0x7f05876527a0>,
..'matrix': (
....matrix([[    1,     2,     3,     4],
....        [   50,    60,    70,    80],
....        [  900,  1000,  1100,  1200],
....        [13000, 14000, 15000, 16000]])
..),
}

For more details on prettier printing, see prettier.py.

ANSI terminal colours🔗

from devtools import sformat, sprint

sprint('this is red', sprint.red)

sprint('this is bold underlined blue on a green background. yuck',
       sprint.blue, sprint.bg_yellow, sprint.bold, sprint.underline)

v = sformat('i am dim', sprint.dim)
print(repr(v))
# > '\x1b[2mi am dim\x1b[0m'

For more details on ansi colours, see ansi.py.

Usage without Import🔗

We all know the annoyance of running code only to discover a missing import, this can be particularly frustrating when the function you're using isn't used except during development.

devtool's debug function can be used without import if you add debug to __builtins__ in sitecustomize.py.

Two ways to do this:

Automatic install🔗

Warning

This is experimental, please create an issue if you encounter any problems.

To install debug into __builtins__ automatically, run:

python -m devtools install

This command won't write to any files, but it should print a command for you to run to add/edit sitecustomize.py.

Manual install🔗

To manually add debug to __builtins__, add the following to sitecustomize.py or any code which is always imported.

import sys
# we don't install here for pytest as it breaks pytest, it is
# installed later by a pytest fixture
if not sys.argv[0].endswith('pytest'):
    import builtins
    try:
        from devtools import debug
    except ImportError:
        pass
    else:
        setattr(builtins, 'debug', debug)

The ImportError exception is important since you'll want python to run fine even if devtools isn't installed.

This approach has another advantage: if you forget to remove debug(...) calls from your code, CI (which won't have devtools installed) should fail both on execution and linting, meaning you don't end up with extraneous debug calls in production code.