python-devtools

pypi license

Current Version: v0.5.1

Dev tools for python.

The debug print command python never had (and other things).

Install

Assuming you have python 3.5+ and pip installed, just:

pip install devtools

If pygments is installed devtools will colourise output to make it even more readable. Chances are you already have pygments installed if you’re using ipython, otherwise it can be installed along with devtools via pip install devtools[pygments].

Debug

Somehow in the 26 years (and counting) of active development of python, no one thought to add a simple and readable way to print stuff during development. (If you know why this is, I’d love to hear).

The wait is over:

from devtools import debug

v1 = {
    'foo': {1: 'nested', 2: 'dict'},
    'bar': ['apple', 'banana', 'carrot', 'grapefruit'],
}

debug(v1, sum(range(5)))

🡣

examples/example1.py:8 <module>
    v1: {
        'foo': {
            1: 'nested',
            2: 'dict',
        },
        'bar': [
            'apple',
            'banana',
            'carrot',
            'grapefruit',
        ],
    } (dict) len=2
    sum(range(5)): 10 (int)

debug is like print on steroids, and coffee, and some orange pills you found down the back of a chair on the tube:

  • each output is prefixed with the file, line number and function where debug was called

  • the variable name or expression being printed is shown

  • each argument is printed “pretty” on a new line, see prettier print

  • if pygments is installed the output is highlighted

A more complex example of debug shows more of what it can do.

from devtools import debug
import numpy as np

foo = {
    'foo': np.array(range(20)),
    'bar': [{'a': i, 'b': {j for j in range(1 + i * 2)}} for i in range(3)],
    'spam': (i for i in ['i', 'am', 'a', 'generator']),
}

debug(foo)

# kwargs can be used as keys for what you are printing
debug(
    long_string='long strings get wrapped ' * 10,
    new_line_string='wraps also on newline\n' * 3,
)

bar = {1: 2, 11: 12}
# debug can also show the output of expressions
debug(
    len(foo),
    bar[1],
    foo == bar
)

🡣

examples/example2.py:10 <module>
    foo: {
        'foo': (
            array([ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, 13, 14, 15, 16,
                   17, 18, 19])
        ),
        'bar': [
            {
                'a': 0,
                'b': {0},
            },
            {
                'a': 1,
                'b': {0, 1, 2},
            },
            {
                'a': 2,
                'b': {
                    0,
                    1,
                    2,
                    3,
                    4,
                },
            },
        ],
        'spam': (
            'i',
            'am',
            'a',
            'generator',
        ),
    } (dict) len=3
examples/example2.py:13 <module>
    long_string: (
        'long strings get wrapped long strings get wrapped long strings get wrapped long strings get wrapped long stri'
        'ngs get wrapped long strings get wrapped long strings get wrapped long strings get wrapped long strings get w'
        'rapped long strings get wrapped '
    ) (str) len=250
    new_line_string: (
        'wraps also on newline\n'
        'wraps also on newline\n'
        'wraps also on newline\n'
    ) (str) len=66
examples/example2.py:20 <module>
    len(foo): 3 (int)
    bar[1]: 2 (int)
    foo == bar: False (bool)

Other debug tools

The debug namespace includes a number of other useful functions:

  • debug.format() same as calling debug() but returns a DebugOutput rather than printing the output

  • debug.timer() returns an instance of devtool’s Timer class suitable for timing code execution

  • debug.breakpoint() introduces a breakpoint using pdb

import random

from devtools import debug

# debug.format() behaves the same as debug() except it returns an object to inspect or print
r = debug.format(x=123, y=321)
print(r)
print(repr(r))

values = list(range(int(1e5)))
# timer can be used as a context manager or directly
with debug.timer('shuffle values'):
    random.shuffle(values)

t2 = debug.timer('sort values').start()
sorted(values)
t2.capture()

# if used repeatedly a summary is available
t3 = debug.timer()
for i in [1e4, 1e6, 1e7]:
    with t3('sum {}'.format(i), verbose=False):
        sum(range(int(i)))

t3.summary(verbose=True)

# debug.breakpoint()
# would drop to a prompt:
# > /python-devtools/docs/examples/more_debug.py(28)<module>()->None
# -> debug.breakpoint()
# (Pdb)

🡣

examples/more_debug.py:6 <module>
    x: 123 (int)
    y: 321 (int)
<DebugOutput examples/more_debug.py:6 <module> arguments: x: 123 (int) y: 321 (int)>
shuffle values: 0.099s elapsed
sort values: 0.045s elapsed
    sum 10000.0: 0.000s elapsed
    sum 1000000.0: 0.021s elapsed
    sum 10000000.0: 0.215s elapsed
3 times: mean=0.079s stdev=0.118s min=0.000s max=0.215s

Prettier print

Python comes with pretty print, problem is quite often it’s not that pretty, it also doesn’t cope well with non standard python objects (think numpy arrays or django querysets) which have their own pretty print functionality.

To get round this devtools comes with prettier print, my take on pretty printing. You can see it in use above in debug(), but it can also be used directly:

from devtools import PrettyFormat, pprint, pformat
import numpy as np

v = {
    'foo': {'whatever': [3, 2, 1]},
    'sentence': 'hello\nworld',
    'generator': (i * 2 for i in [1, 2, 3]),
    'matrix': np.matrix([[1, 2, 3, 4],
                         [50, 60, 70, 80],
                         [900, 1000, 1100, 1200],
                         [13000, 14000, 15000, 16000]])
}

# pretty print of v
pprint(v)

# as above without colours, the generator will also be empty as it's already been evaluated
s = pformat(v, highlight=False)
print(s)

pp = PrettyFormat(
     indent_step=2,  # default: 4
     indent_char='.',  # default: space
     repr_strings=True,  # default: False
     simple_cutoff=2,  # default: 10 (if repr is below this length it'll be shown on one line)
     width=80,  # default: 120
     yield_from_generators=False  # default: True (whether to evaluate generators)
)

print(pp(v, highlight=True))

🡣

{
    'foo': {
        'whatever': [3, 2, 1],
    },
    'sentence': (
        'hello\n'
        'world'
    ),
    'generator': (
        2,
        4,
        6,
    ),
    'matrix': (
        matrix([[    1,     2,     3,     4],
                [   50,    60,    70,    80],
                [  900,  1000,  1100,  1200],
                [13000, 14000, 15000, 16000]])
    ),
}
{
    'foo': {
        'whatever': [3, 2, 1],
    },
    'sentence': (
        'hello\n'
        'world'
    ),
    'generator': (
    ),
    'matrix': (
        matrix([[    1,     2,     3,     4],
                [   50,    60,    70,    80],
                [  900,  1000,  1100,  1200],
                [13000, 14000, 15000, 16000]])
    ),
}
{
..'foo': {
....'whatever': [
......3,
......2,
......1,
....],
..},
..'sentence': 'hello\nworld',
..'generator': <generator object <genexpr> at 0x7f0fe5789308>,
..'matrix': (
....matrix([[    1,     2,     3,     4],
....        [   50,    60,    70,    80],
....        [  900,  1000,  1100,  1200],
....        [13000, 14000, 15000, 16000]])
..),
}

For more details on prettier printing, see prettier.py.

ANSI terminal colours

from devtools import sprint, sformat

sprint('this is red', sprint.red)

sprint('this is bold underlined blue on a green background. yuck',
       sprint.blue, sprint.bg_yellow, sprint.bold, sprint.underline)

v = sformat('i am dim', sprint.dim)
print(repr(v))
# > '\x1b[2mi am dim\x1b[0m'

For more details on ansi colours, see ansi.py.

Usage without import

We all know the annoyance of running code only to discover a missing import, this can be particularly frustrating when the function you’re using isn’t used except during development.

You can setup your environment to make debug available at all times by editing sitecustomize.py, with ubuntu and python3.6 this file can be found at /usr/lib/python3.6/sitecustomize.py but you might need to look elsewhere depending on your OS/python version.

Add the following to sitecustomize.py

...

try:
    from devtools import debug
except ImportError:
    pass
else:
    __builtins__['debug'] = debug

The ImportError exception is important since you’ll want python to run fine even if devtools isn’t installed.

This approach has another advantage: if you forget to remove debug(...) calls from your code, CI (which won’t have devtools installed) should fail both on execution and linting, meaning you don’t end up with extraneous debug calls in production code.

History

v0.5.1 (2019-10-09)

  • fix python tag in setup.cfg, #46

v0.5.0 (2019-01-03)

  • support MultiDict, #34

  • support __pretty__ method, #36

v0.4.0 (2018-12-29)

  • remove use of warnings, include in output, #30

  • fix rendering errors #31

  • better str and bytes wrapping #32

  • add len everywhere possible, part of #16

v0.3.0 (2017-10-11)

  • allow async/await arguments

  • fix subscript

  • fix weird named tuples eg. mock > call_args

  • add timer

v0.2.0 (2017-09-14)

  • improve output

  • numerous bug fixes