Narrative Documentation¶
unittest_expander is a Python library that provides flexible and
easy-to-use tools to parametrize your unit tests, especially (but not
limited to) those based on unittest.TestCase from the Python
standard library.
The unittest_expander module provides the following tools:
- a test class decorator:
expand(), - a test method decorator:
foreach(), - and a few helpers:
param,paramseqandcurrent.
Let’s see how to use them…
Basic use of expand() and foreach()¶
Assume we have a function that checks whether the given number is even or not:
>>> def is_even(n):
... return n % 2 == 0
Of course, in the real world the code we write is usually more interesting… Anyway, most often we want to test how it works for different parameters. At the same time, it is not the best idea to test many cases in a loop within one test method – because of lack of test isolation (tests depend on other ones – they may inherit some state which can affect their results), less information on failures (a test failure or fixture-related error may prevent subsequent tests from being run), less clear fail messages (when you don’t see at first glance which case is the actual culprit), etc.
So let’s write our tests in a smarter way:
>>> import unittest
>>> from unittest_expander import expand, foreach
>>>
>>> @expand
... class Test_is_even(unittest.TestCase):
...
... @foreach(0, 2, -14) # call variant #1: parameter collection
... def test_even(self, n): # specified using multiple arguments
... self.assertTrue(is_even(n))
...
... @foreach([-1, 17]) # call variant #2: parameter collection as
... def test_odd(self, n): # one argument being a container (e.g. list)
... self.assertFalse(is_even(n))
...
... # just to demonstrate that test cases are really isolated
... def setUp(self):
... sys.stdout.write(' [DEBUG: separate test setUp] ')
... sys.stdout.flush()
As you see, it’s fairly simple: you bind parameter collections to your
test methods with the foreach() decorator and decorate the whole
test class with the expand() decorator. The latter does the
actual job, i.e., generates (and adds to the test class) parametrized
versions of the test methods.
Let’s run this stuff…
>>> # a helper function that will run tests in our examples --
>>> # NORMALLY YOU DON'T NEED IT, of course!
>>> import sys
>>> def run_tests(*test_classes):
... suite = unittest.TestSuite(
... unittest.TestLoader().loadTestsFromTestCase(cls)
... for cls in test_classes)
... unittest.TextTestRunner(stream=sys.stdout, verbosity=2).run(suite)
...
>>> run_tests(Test_is_even) # doctest: +ELLIPSIS
test_even__<-14> ... [DEBUG: separate test setUp] ok
test_even__<0> ... [DEBUG: separate test setUp] ok
test_even__<2> ... [DEBUG: separate test setUp] ok
test_odd__<-1> ... [DEBUG: separate test setUp] ok
test_odd__<17> ... [DEBUG: separate test setUp] ok
...Ran 5 tests...
OK
To test our is_even() function we created two test methods – each accepting one parameter value.
Another approach could be to define a method that accepts a couple of arguments:
>>> @expand
... class Test_is_even(unittest.TestCase):
...
... @foreach(
... (-14, True),
... (-1, False),
... (0, True),
... (2, True),
... (17, False),
... )
... def test(self, n, expected):
... actual = is_even(n)
... self.assertTrue(isinstance(actual, bool))
... self.assertEqual(actual, expected)
...
>>> run_tests(Test_is_even) # doctest: +ELLIPSIS
test__<-1,False> ... ok
test__<-14,True> ... ok
test__<0,True> ... ok
test__<17,False> ... ok
test__<2,True> ... ok
...Ran 5 tests...
OK
As you see, you can use a tuple to specify several parameter values for a test call.
More flexibility: param¶
Parameters can be specified in a more descriptive way – in particular,
using also keyword arguments. It is possible when you use param
objects instead of tuples:
>>> from unittest_expander import param
>>>
>>> @expand
... class Test_is_even(unittest.TestCase):
...
... @foreach(
... param(-14, expected=True),
... param(-1, expected=False),
... param(0, expected=True),
... param(2, expected=True),
... param(17, expected=False),
... )
... def test(self, n, expected):
... actual = is_even(n)
... self.assertTrue(isinstance(actual, bool))
... self.assertEqual(actual, expected)
...
>>> run_tests(Test_is_even) # doctest: +ELLIPSIS
test__<-1,expected=False> ... ok
test__<-14,expected=True> ... ok
test__<0,expected=True> ... ok
test__<17,expected=False> ... ok
test__<2,expected=True> ... ok
...Ran 5 tests...
OK
Generated labels of our tests (attached to the names of the generated test methods) became less cryptic. But what to do if we need to label our parameters explicitly?
We can use the label() method of param objects:
>>> @expand
... class Test_is_even(unittest.TestCase):
...
... @foreach(
... param(sys.maxsize, expected=False).label('sys.maxsize'),
... param(-sys.maxsize, expected=False).label('-sys.maxsize'),
... )
... def test(self, n, expected):
... actual = is_even(n)
... self.assertTrue(isinstance(actual, bool))
... self.assertEqual(actual, expected)
...
>>> run_tests(Test_is_even) # doctest: +ELLIPSIS
test__<-sys.maxsize> ... ok
test__<sys.maxsize> ... ok
...Ran 2 tests...
OK
Note that the label (either auto-generated from parameter values or
explicitly specified with param.label()) is available within the
test method definition as current.label (current is a
special object described later):
>>> @expand
... class Test_is_even(unittest.TestCase):
...
... @foreach(
... param(sys.maxsize, expected=False).label('sys.maxsize'),
... param(-sys.maxsize, expected=False).label('-sys.maxsize'),
... )
... def test(self, n, expected):
... actual = is_even(n)
... self.assertTrue(isinstance(actual, bool))
... self.assertEqual(actual, expected)
... assert current.label in ('sys.maxsize', '-sys.maxsize')
... sys.stdout.write(' [DEBUG: {!r}] '.format(current.label))
... sys.stdout.flush()
...
>>> run_tests(Test_is_even) # doctest: +ELLIPSIS
test__<-sys.maxsize> ... [DEBUG: '-sys.maxsize'] ok
test__<sys.maxsize> ... [DEBUG: 'sys.maxsize'] ok
...Ran 2 tests...
OK
Other ways to explicitly label your tests¶
You can also label particular tests by passing a dictionary directly
into foreach():
>>> @expand
... class Test_is_even(unittest.TestCase):
...
... @foreach({
... 'non-integer': (1.2345, False),
... 'horrible abuse!': ('%s', False),
... })
... def test(self, n, expected):
... actual = is_even(n)
... self.assertTrue(isinstance(actual, bool))
... self.assertEqual(actual, expected)
... assert current.label in ('non-integer', 'horrible abuse!')
... sys.stdout.write(' [DEBUG: {!r}] '.format(current.label))
... sys.stdout.flush()
...
>>> run_tests(Test_is_even) # doctest: +ELLIPSIS
test__<horrible abuse!> ... [DEBUG: 'horrible abuse!'] ok
test__<non-integer> ... [DEBUG: 'non-integer'] ok
...Ran 2 tests...
OK
…or just using keyword arguments:
>>> @expand
... class Test_is_even(unittest.TestCase):
...
... @foreach(
... noninteger=(1.2345, False),
... horrible_abuse=('%s', False),
... )
... def test(self, n, expected):
... actual = is_even(n)
... self.assertTrue(isinstance(actual, bool))
... self.assertEqual(actual, expected)
... assert current.label in ('noninteger', 'horrible_abuse')
... sys.stdout.write(' [DEBUG: {!r}] '.format(current.label))
... sys.stdout.flush()
...
>>> run_tests(Test_is_even) # doctest: +ELLIPSIS
test__<horrible_abuse> ... [DEBUG: 'horrible_abuse'] ok
test__<noninteger> ... [DEBUG: 'noninteger'] ok
...Ran 2 tests...
OK
Smart parameter collection: paramseq¶
How to concatenate some separately created parameter collections?
Just transform them (or at least the first of them) into
paramseq instances – and then add one to another
(with the + operator):
>>> from unittest_expander import paramseq
>>>
>>> @expand
... class Test_is_even(unittest.TestCase):
...
... basic_params1 = paramseq( # init variant #1: several parameters
... param(-14, expected=True),
... param(-1, expected=False),
... )
... basic_params2 = paramseq([ # init variant #2: one parameter collection
... param(0, expected=True).label('just zero, because why not?'),
... param(2, expected=True),
... param(17, expected=False),
... ])
... basic = basic_params1 + basic_params2
...
... huge = paramseq({ # explicit labelling by passing a dict
... 'sys.maxsize': param(sys.maxsize, expected=False),
... '-sys.maxsize': param(-sys.maxsize, expected=False),
... })
...
... other = paramseq(
... (-15, False),
... param(15, expected=False),
... # explicit labelling with keyword arguments:
... noninteger=param(1.2345, expected=False),
... horribleabuse=param('%s', expected=False),
... )
...
... just_dict = {
... '18->True': (18, True),
... }
...
... just_list = [
... param(12399999999999999, False),
... param(n=12399999999999998, expected=True),
... ]
...
... # just add them one to another (producing a new paramseq)
... all_params = basic + huge + other + just_dict + just_list
...
... @foreach(all_params)
... def test(self, n, expected):
... actual = is_even(n)
... self.assertTrue(isinstance(actual, bool))
... self.assertEqual(actual, expected)
...
>>> run_tests(Test_is_even) # doctest: +ELLIPSIS
test__<-1,expected=False> ... ok
test__<-14,expected=True> ... ok
test__<-15,False> ... ok
test__<-sys.maxsize> ... ok
test__<15,expected=False> ... ok
test__<17,expected=False> ... ok
test__<18->True> ... ok
test__<2,expected=True> ... ok
test__<<12399999999...>,False> ... ok
test__<expected=True,n=<12399999999...>> ... ok
test__<horribleabuse> ... ok
test__<just zero, because why not?> ... ok
test__<noninteger> ... ok
test__<sys.maxsize> ... ok
...Ran 14 tests...
OK
Note
Parameter collections – being sequences (e.g., list
instances), or mappings (e.g., dict instances), or sets
(e.g., set or frozenset instances), or callable
objects (see below…), or just ready paramseq instances –
do not need to be created or bound within the test class body; you
could, for example, import them from a separate module. Obviously,
that makes data/code reuse and refactorization easier.
Also, note that the call signatures of foreach() and the
paramseq constructor are identical: you pass in either
exactly one positional argument which is a parameter collection, or
several (more than one) positional and/or keyword arguments being
param instances or tuples of parameter values, or singular
parameter values.
Note
We said that a parameter collection can be a sequence (among
others; see the note above). To be more precise: it can be a
sequence, except that it cannot be: a tuple, a text
string (str in Python 3, str or unicode
in Python 2) or a Python 3 binary string-like sequence
(bytes or bytearray).
A paramseq instance can also be created from a callable object
(e.g., a function) that returns a sequence or another iterable
object (e.g., a generator iterator):
>>> from random import randint
>>>
>>> @paramseq # <- yes, used as a decorator
... def randomized(test_cls):
... print('DEBUG: LO = {0.LO}; HI = {0.HI}'.format(test_cls))
... print('----')
... yield param(randint(test_cls.LO, test_cls.HI) * 2,
... expected=True).label('random even')
... yield param(randint(test_cls.LO, test_cls.HI) * 2 + 1,
... expected=False).label('random odd')
...
>>> @expand
... class Test_is_even(unittest.TestCase):
...
... LO = -100
... HI = 100
...
... @foreach(randomized)
... def test_even(self, n, expected):
... actual = is_even(n)
... self.assertTrue(isinstance(actual, bool))
... self.assertEqual(actual, expected)
...
... # reusing the same instance of paramseq to show that the underlying
... # callable is called separately for each use of @foreach:
... @foreach(randomized)
... def test_not_even_when_incremented(self, n, expected):
... actual = (not is_even(n + 1))
... self.assertTrue(isinstance(actual, bool))
... self.assertEqual(actual, expected)
...
DEBUG: LO = -100; HI = 100
----
DEBUG: LO = -100; HI = 100
----
>>> run_tests(Test_is_even) # doctest: +ELLIPSIS
test_even__<random even> ... ok
test_even__<random odd> ... ok
test_not_even_when_incremented__<random even> ... ok
test_not_even_when_incremented__<random odd> ... ok
...Ran 4 tests...
OK
A callable object (such as the generator function in the example
above) which is passed to the paramseq’s constructor (or
directly to foreach()) should accept either no arguments or one
positional argument – in the latter case the test class will be
passed in.
Note
The callable object will be called – and its iterable result will
be iterated over (consumed) – when the expand() decorator
is being executed, directly before generating parametrized test
methods.
What should also be emphasized is that those operations (the
aforementioned call and iterating over its result) will be
performed separately for each use of foreach() with our
paramseq instance as its argument (or with another
paramseq instance that includes our instance; see the
following code snippet in which the input_values_and_results
instance includes the previously created randomized instance).
>>> @expand
... class Test_is_even(unittest.TestCase):
...
... LO = -999999
... HI = 999999
...
... # reusing the same, previously created, instance of paramseq
... # (`randomized`) to show that the underlying callable will
... # still be called separately for each use of @foreach...
... input_values_and_results = randomized + [
... param(-14, expected=True),
... param(-1, expected=False),
... param(0, expected=True),
... param(2, expected=True),
... param(17, expected=False),
... ]
...
... @foreach(input_values_and_results)
... def test_even(self, n, expected):
... actual = is_even(n)
... self.assertTrue(isinstance(actual, bool))
... self.assertEqual(actual, expected)
...
... @foreach(input_values_and_results)
... def test_not_even_when_incremented(self, n, expected):
... actual = (not is_even(n + 1))
... self.assertTrue(isinstance(actual, bool))
... self.assertEqual(actual, expected)
...
DEBUG: LO = -999999; HI = 999999
----
DEBUG: LO = -999999; HI = 999999
----
>>> run_tests(Test_is_even) # doctest: +ELLIPSIS
test_even__<-1,expected=False> ... ok
test_even__<-14,expected=True> ... ok
test_even__<0,expected=True> ... ok
test_even__<17,expected=False> ... ok
test_even__<2,expected=True> ... ok
test_even__<random even> ... ok
test_even__<random odd> ... ok
test_not_even_when_incremented__<-1,expected=False> ... ok
test_not_even_when_incremented__<-14,expected=True> ... ok
test_not_even_when_incremented__<0,expected=True> ... ok
test_not_even_when_incremented__<17,expected=False> ... ok
test_not_even_when_incremented__<2,expected=True> ... ok
test_not_even_when_incremented__<random even> ... ok
test_not_even_when_incremented__<random odd> ... ok
...Ran 14 tests...
OK
Combining several foreach() to get Cartesian product¶
You can stack up two or more foreach() decorators to the same test
method – to combine several parameter collections, obtaining the
Cartesian product of them:
>>> @expand
... class Test_is_even(unittest.TestCase):
...
... # one param collection (7 items)
... @paramseq
... def randomized():
... yield param(randint(-(10 ** 6), 10 ** 6) * 2,
... expected=True).label('random even')
... yield param(randint(-(10 ** 6), 10 ** 6) * 2 + 1,
... expected=False).label('random odd')
... input_values_and_results = randomized + [ # (<- note the use of +)
... param(-14, expected=True),
... param(-1, expected=False),
... param(0, expected=True),
... param(2, expected=True),
... param(17, expected=False),
... ]
...
... # another param collection (2 items)
... input_types = dict(
... integer=int,
... floating=float,
... )
...
... # let's combine them (7 * 2 -> 14 parametrized tests)
... @foreach(input_values_and_results)
... @foreach(input_types)
... def test(self, input_type, n, expected):
... n = input_type(n)
... actual = is_even(n)
... self.assertTrue(isinstance(actual, bool))
... self.assertEqual(actual, expected)
...
>>> run_tests(Test_is_even) # doctest: +ELLIPSIS
test__<floating, -1,expected=False> ... ok
test__<floating, -14,expected=True> ... ok
test__<floating, 0,expected=True> ... ok
test__<floating, 17,expected=False> ... ok
test__<floating, 2,expected=True> ... ok
test__<floating, random even> ... ok
test__<floating, random odd> ... ok
test__<integer, -1,expected=False> ... ok
test__<integer, -14,expected=True> ... ok
test__<integer, 0,expected=True> ... ok
test__<integer, 17,expected=False> ... ok
test__<integer, 2,expected=True> ... ok
test__<integer, random even> ... ok
test__<integer, random odd> ... ok
...Ran 14 tests...
OK
If parameters combined this way specify some conflicting keyword arguments, they are detected and an error is reported:
>>> params1 = [param(a=1, b=2, c=3)]
>>> params2 = [param(b=4, c=3, d=2)]
>>>
>>> @expand # doctest: +ELLIPSIS
... class TestSomething(unittest.TestCase):
...
... @foreach(params2)
... @foreach(params1)
... def test(self, **kw):
... "something"
...
Traceback (most recent call last):
...
ValueError: conflicting keyword arguments: 'b', 'c'
Context-manager-based fixtures: param.context()¶
When dealing with resources managed with context managers, you can
specify a context manager factory and its arguments using the
context() method of a param object – then each
call of the resultant parametrized test will be enclosed in a dedicated
context manager instance (created by calling the context manager
factory with the given arguments).
XXX TODO
>>> from tempfile import NamedTemporaryFile
>>>
>>> @expand
... class TestSaveLoad(unittest.TestCase):
...
... params_with_contexts = [
... param(save='', load='').context(NamedTemporaryFile, 'w+t'),
... param(save='abc', load='abc').context(NamedTemporaryFile, 'w+t'),
... ]
...
... @foreach(params_with_contexts)
... def test_save_load(self, save, load):
... file = current.context_targets[0]
... file.write(save)
... file.seek(0)
... load_actually = file.read()
... self.assertEqual(load_actually, load)
...
... # reusing the same params to show that a *new* context manager
... # instance is created for each test call:
... @foreach(params_with_contexts)
... def test_save_load_with_spaces(self, save, load):
... file = current.context_targets[0]
... file.write(' ' + save + ' ')
... file.seek(0)
... load_actually = file.read()
... self.assertEqual(load_actually, ' ' + load + ' ')
...
>>> run_tests(TestSaveLoad) # doctest: +ELLIPSIS
test_save_load__<load='',save=''> ... ok
test_save_load__<load='abc',save='abc'> ... ok
test_save_load_with_spaces__<load='',save=''> ... ok
test_save_load_with_spaces__<load='abc',save='abc'> ... ok
...Ran 4 tests...
OK
>>>
>>> # repeating the tests to show that, really, a *new* context manager
... # instance is created for *each* test call:
... run_tests(TestSaveLoad) # doctest: +ELLIPSIS
test_save_load__<load='',save=''> ... ok
test_save_load__<load='abc',save='abc'> ... ok
test_save_load_with_spaces__<load='',save=''> ... ok
test_save_load_with_spaces__<load='abc',save='abc'> ... ok
...Ran 4 tests...
OK
As you can see in the above example, a sequence of context manager
as-targets (i.e., objects returned by context managers’
__enter__()) is available within the test method definition as
current.context_targets (current is a special object
described later):
current.context_targets is a sequence because there can be more
than one context per parameter collection’s item, e.g.:
>>> import contextlib
>>> @contextlib.contextmanager
... def debug_cm(tag=None):
... debug.append('enter' + (':{}'.format(tag) if tag else ''))
... yield tag
... debug.append('exit' + (':{}'.format(tag) if tag else ''))
...
>>> debug = []
>>>
>>> @expand
... class TestSaveLoad(unittest.TestCase):
...
... params_with_contexts = [
... (
... param(save='', load='', expected_tag='FOO')
... .context(NamedTemporaryFile, 'w+t') # (outer one)
... .context(debug_cm, tag='FOO') # (inner one)
... ),
... (
... param(save='abc', load='abc', expected_tag='BAR')
... .context(NamedTemporaryFile, 'w+t')
... .context(debug_cm, tag='BAR')
... ),
... ]
...
... @foreach(params_with_contexts)
... def test_save_load(self, save, load, expected_tag):
... file, tag = current.context_targets
... assert tag == expected_tag
... file.write(save)
... file.seek(0)
... load_actually = file.read()
... self.assertEqual(load_actually, load)
... debug.append('test')
...
>>> debug == []
True
>>> run_tests(TestSaveLoad) # doctest: +ELLIPSIS
test_save_load__<expected_tag='BAR',load='abc',save='abc'> ... ok
test_save_load__<expected_tag='FOO',load='',save=''> ... ok
...Ran 2 tests...
OK
>>> debug == [
... 'enter:BAR', 'test', 'exit:BAR',
... 'enter:FOO', 'test', 'exit:FOO',
... ]
True
Contexts are properly handled (context managers’ __enter__() and
__exit__() are properly called…) also when errors occur (with
some legitimate subtle reservations – see:
Contexts cannot suppress exceptions unless you enable that explicitly):
>>> class ErrDebugCM(object):
...
... def __init__(self, tag):
... debug.append('init:' + tag)
... self._tag = tag
...
... def __enter__(self):
... if self._tag.endswith('context-enter-error'):
... debug.append('ERR-enter:' + self._tag)
... raise RuntimeError('error in __enter__')
... debug.append('enter:' + self._tag)
... return self._tag
...
... def __exit__(self, exc_type, exc_val, exc_tb):
... if exc_type is None:
... if self._tag.endswith('context-exit-error'):
... debug.append('ERR-exit:' + self._tag)
... raise RuntimeError('error in __exit__')
... debug.append('exit:' + self._tag)
... else:
... debug.append('ERR-exit:' + self._tag)
...
>>> debug = []
>>> err_params = [
... (
... param().label('no_error')
... .context(ErrDebugCM, tag='outer')
... .context(ErrDebugCM, tag='inner')
... ),
... (
... param().label('test_fail')
... .context(ErrDebugCM, tag='outer')
... .context(ErrDebugCM, tag='inner')
... ),
... (
... param().label('test_error')
... .context(ErrDebugCM, tag='outer')
... .context(ErrDebugCM, tag='inner')
... ),
... (
... param().label('inner_context_enter_error')
... .context(ErrDebugCM, tag='outer')
... .context(ErrDebugCM, tag='inner-context-enter-error')
... ),
... (
... param().label('inner_context_exit_error')
... .context(ErrDebugCM, tag='outer')
... .context(ErrDebugCM, tag='inner-context-exit-error')
... ),
... (
... param().label('outer_context_enter_error')
... .context(ErrDebugCM, tag='outer-context-enter-error')
... .context(ErrDebugCM, tag='inner')
... ),
... (
... param().label('outer_context_exit_error')
... .context(ErrDebugCM, tag='outer-context-exit-error')
... .context(ErrDebugCM, tag='inner')
... ),
... ]
>>>
>>> @expand
... class SillyTest(unittest.TestCase):
...
... def setUp(self):
... debug.append('setUp')
...
... def tearDown(self):
... debug.append('tearDown')
...
... @foreach(err_params)
... def test(self):
... if current.label == 'test_fail':
... debug.append('FAIL-test')
... self.fail()
... elif current.label == 'test_error':
... debug.append('ERROR-test')
... raise RuntimeError
... else:
... debug.append('test')
...
>>> run_tests(SillyTest) # doctest: +ELLIPSIS
test__<inner_context_enter_error> ... ERROR
test__<inner_context_exit_error> ... ERROR
test__<no_error> ... ok
test__<outer_context_enter_error> ... ERROR
test__<outer_context_exit_error> ... ERROR
test__<test_error> ... ERROR
test__<test_fail> ... FAIL
...Ran 7 tests...
FAILED (failures=1, errors=5)
>>> debug == [
... # inner_context_enter_error
... 'setUp',
... 'init:outer',
... 'enter:outer',
... 'init:inner-context-enter-error',
... 'ERR-enter:inner-context-enter-error',
... 'ERR-exit:outer',
... 'tearDown',
...
... # inner_context_exit_error
... 'setUp',
... 'init:outer',
... 'enter:outer',
... 'init:inner-context-exit-error',
... 'enter:inner-context-exit-error',
... 'test',
... 'ERR-exit:inner-context-exit-error',
... 'ERR-exit:outer',
... 'tearDown',
...
... # no_error
... 'setUp',
... 'init:outer',
... 'enter:outer',
... 'init:inner',
... 'enter:inner',
... 'test',
... 'exit:inner',
... 'exit:outer',
... 'tearDown',
...
... # outer_context_enter_error
... 'setUp',
... 'init:outer-context-enter-error',
... 'ERR-enter:outer-context-enter-error',
... 'tearDown',
...
... # outer_context_exit_error
... 'setUp',
... 'init:outer-context-exit-error',
... 'enter:outer-context-exit-error',
... 'init:inner',
... 'enter:inner',
... 'test',
... 'exit:inner',
... 'ERR-exit:outer-context-exit-error',
... 'tearDown',
...
... # test_error
... 'setUp',
... 'init:outer',
... 'enter:outer',
... 'init:inner',
... 'enter:inner',
... 'ERROR-test',
... 'ERR-exit:inner',
... 'ERR-exit:outer',
... 'tearDown',
...
... # test_fail
... 'setUp',
... 'init:outer',
... 'enter:outer',
... 'init:inner',
... 'enter:inner',
... 'FAIL-test',
... 'ERR-exit:inner',
... 'ERR-exit:outer',
... 'tearDown',
... ]
True
Note that contexts are handled directly before (by running
__enter__()) and after (by running __exit__()) each relevant
test method call, that is, after the setUp() and before
tearDown() calls (if those unittest.TestCase-specific
methods are engaged) – so setUp() and tearDown() are
not affected by any errors related to those contexts.
Obviously, if an error in setUp() occurs then the test method is
not called at all. Therefore, then, relevant context managers are not
even created:
>>> def setUp(self):
... debug.append('setUp')
... raise ValueError
...
>>> SillyTest.setUp = setUp
>>> debug = []
>>> run_tests(SillyTest) # doctest: +ELLIPSIS
test__<inner_context_enter_error> ... ERROR
test__<inner_context_exit_error> ... ERROR
test__<no_error> ... ERROR
test__<outer_context_enter_error> ... ERROR
test__<outer_context_exit_error> ... ERROR
test__<test_error> ... ERROR
test__<test_fail> ... ERROR
...Ran 7 tests...
FAILED (errors=7)
>>> debug == ['setUp', 'setUp', 'setUp', 'setUp', 'setUp', 'setUp', 'setUp']
True
Convenience shortcut: paramseq.context()¶
You can use the method paramseq.context() to apply the given
context properties to all parameter items the paramseq
instance aggregates:
>>> @contextlib.contextmanager
... def silly_cm():
... yield 42
...
>>> @expand
... class TestSaveLoad(unittest.TestCase):
...
... params_with_contexts = paramseq(
... param(save='', load=''),
... param(save='abc', load='abc'),
... ).context(NamedTemporaryFile, 'w+t').context(silly_cm)
...
... @foreach(params_with_contexts)
... def test_save_load(self, save, load):
... file, silly_cm_target = current.context_targets
... assert silly_cm_target == 42
... file.write(save)
... file.seek(0)
... load_actually = file.read()
... self.assertEqual(load_actually, load)
...
>>> run_tests(TestSaveLoad) # doctest: +ELLIPSIS
test_save_load__<load='',save=''> ... ok
test_save_load__<load='abc',save='abc'> ... ok
...Ran 2 tests...
OK
It should be noted that paramseq.context() as well as
param.context() and param.label() methods create new objects
(respectively paramseq or param instances), without
modifying the existing ones.
>>> pseq1 = paramseq(1, 2, 3)
>>> pseq2 = pseq1.context(open, '/etc/hostname', 'rb')
>>> isinstance(pseq1, paramseq) and isinstance(pseq2, paramseq)
True
>>> pseq1 is not pseq2
True
>>> p1 = param(1, 2, c=3)
>>> p2 = p1.context(open, '/etc/hostname', 'rb')
>>> p3 = p2.label('one with label')
>>> isinstance(p1, param) and isinstance(p2, param) and isinstance(p3, param)
True
>>> p1 is not p2
True
>>> p2 is not p3
True
>>> p3 is not p1
True
Generally, instances of these types (param and paramseq)
should be considered immutable.
Contexts cannot suppress exceptions unless you enable that explicitly¶
The Python context manager protocol provides a way to suppress an
exception occuring in the code enclosed by a context: the exception is
suppresed (not propagated) if the context manager’s
__exit__() method returns a true value (such as True).
It does not apply to context managers declared with
param.context() or paramseq.context(): if __exit__()
of such a context manager returns a true value, it is ignored and
the exception (if any) is propagated anyway. The rationale of this
behavior is that silencing exceptions is generally not a good idea
when dealing with testing (it could easily make your tests leaky and
useless).
However, if you really need to allow your context manager to
suppress exceptions, pass the keyword argument _enable_exc_suppress_=True
(note the single underscores at the beginning and the end of its name)
to the param.context() or paramseq.context() method (and,
of course, make the __exit__() context manager’s method return a
true value).
Here we pass _enable_exc_suppress_=True to param.context():
>>> class SillySuppressingCM(object):
... def __enter__(self): return self
... def __exit__(self, exc_type, exc_val, exc_tb):
... if exc_type is not None:
... debug.append('suppressing {}'.format(exc_type.__name__))
... return True # suppress any exception
...
>>> @expand
... class SillyExcTest(unittest.TestCase):
...
... @foreach(
... param(test_error=AssertionError)
... .context(SillySuppressingCM, _enable_exc_suppress_=True),
... param(test_error=KeyError)
... .context(SillySuppressingCM, _enable_exc_suppress_=True),
... )
... def test_it(self, test_error):
... debug.append('raising {}'.format(test_error.__name__))
... raise test_error('ha!')
...
>>> debug = []
>>> run_tests(SillyExcTest) # doctest: +ELLIPSIS
test_it__... ok
test_it__... ok
...Ran 2 tests...
OK
>>> debug == [
... 'raising AssertionError',
... 'suppressing AssertionError',
... 'raising KeyError',
... 'suppressing KeyError',
... ]
True
Here we pass _enable_exc_suppress_=True to paramseq.context():
>>> my_params = paramseq(
... AssertionError,
... KeyError,
... ).context(SillySuppressingCM, _enable_exc_suppress_=True)
>>> @expand
... class SecondSillyExcTest(unittest.TestCase):
...
... @foreach(my_params)
... def test_it(self, test_error):
... debug.append('raising {}'.format(test_error.__name__))
... raise test_error('ha!')
...
>>> debug = []
>>> run_tests(SecondSillyExcTest) # doctest: +ELLIPSIS
test_it__... ok
test_it__... ok
...Ran 2 tests...
OK
>>> debug == [
... 'raising AssertionError',
... 'suppressing AssertionError',
... 'raising KeyError',
... 'suppressing KeyError',
... ]
True
Yet another example:
>>> class ErrorCM:
... def __init__(self, error): self.error = error
... def __enter__(self): return self
... def __exit__(self, exc_type, exc_val, exc_tb):
... if exc_type is not None:
... debug.append('replacing {} with {}'.format(
... exc_type.__name__, self.error.__name__))
... else:
... debug.append('raising {}'.format(self.error.__name__))
... raise self.error('argh!')
...
>>> @expand
... class AnotherSillyExcTest(unittest.TestCase):
...
... @foreach([
... param(test_error=AssertionError)
... .context(SillySuppressingCM, _enable_exc_suppress_=True),
... param(test_error=KeyError)
... .context(SillySuppressingCM, _enable_exc_suppress_=True)
... .context(ErrorCM, error=RuntimeError),
... param(test_error=None)
... .context(SillySuppressingCM, _enable_exc_suppress_=True),
... param(test_error=None)
... .context(SillySuppressingCM, _enable_exc_suppress_=True)
... .context(ErrorCM, error=IndexError, _enable_exc_suppress_=True)
... .context(ErrorCM, error=TypeError, _enable_exc_suppress_=True),
... param(test_error=OSError)
... .context(SillySuppressingCM, _enable_exc_suppress_=True)
... .context(ErrorCM, error=ValueError)
... .context(ErrorCM, error=ZeroDivisionError),
... param(test_error=UnboundLocalError)
... .context(SillySuppressingCM, _enable_exc_suppress_=True)
... .context(ErrorCM, error=ValueError)
... .context(SillySuppressingCM, _enable_exc_suppress_=True)
... .context(ErrorCM, error=ZeroDivisionError)
... .context(SillySuppressingCM, _enable_exc_suppress_=True),
... ])
... def test_it(self, test_error):
... if test_error is None:
... debug.append('no error')
... else:
... debug.append('raising {}'.format(test_error.__name__))
... raise test_error('ha!')
...
>>> debug = []
>>> run_tests(AnotherSillyExcTest) # doctest: +ELLIPSIS
test_it__... ok
test_it__... ok
test_it__... ok
test_it__... ok
test_it__... ok
test_it__... ok
...Ran 6 tests...
OK
>>> debug == [
... 'raising AssertionError',
... 'suppressing AssertionError',
...
... 'raising KeyError',
... 'replacing KeyError with RuntimeError',
... 'suppressing RuntimeError',
...
... 'raising OSError',
... 'replacing OSError with ZeroDivisionError',
... 'replacing ZeroDivisionError with ValueError',
... 'suppressing ValueError',
...
... 'raising UnboundLocalError',
... 'suppressing UnboundLocalError',
... 'raising ZeroDivisionError',
... 'suppressing ZeroDivisionError',
... 'raising ValueError',
... 'suppressing ValueError',
...
... 'no error',
...
... 'no error',
... 'raising TypeError',
... 'replacing TypeError with IndexError',
... 'suppressing IndexError',
... ]
True
Normally – without _enable_exc_suppress_=True – exceptions
are propagated even when __exit__() returns a true value:
>>> @expand
... class AnotherSillyExcTest2(unittest.TestCase):
...
... @foreach([
... param(test_error=AssertionError)
... .context(SillySuppressingCM),
... param(test_error=KeyError)
... .context(SillySuppressingCM)
... .context(ErrorCM, error=RuntimeError),
... param(test_error=None)
... .context(SillySuppressingCM),
... param(test_error=None)
... .context(SillySuppressingCM)
... .context(ErrorCM, error=IndexError)
... .context(ErrorCM, error=TypeError),
... param(test_error=OSError)
... .context(SillySuppressingCM)
... .context(ErrorCM, error=ValueError)
... .context(ErrorCM, error=ZeroDivisionError),
... param(test_error=UnboundLocalError)
... .context(SillySuppressingCM)
... .context(ErrorCM, error=ValueError)
... .context(SillySuppressingCM)
... .context(ErrorCM, error=ZeroDivisionError)
... .context(SillySuppressingCM),
... ])
... def test_it(self, test_error):
... if test_error is None:
... debug.append('no error')
... else:
... debug.append('raising {}'.format(test_error.__name__))
... raise test_error('ha!')
...
>>> debug = []
>>> run_tests(AnotherSillyExcTest2) # doctest: +ELLIPSIS
test_it__... FAIL
test_it__... ERROR
test_it__... ERROR
test_it__... ERROR
test_it__... ok
test_it__... ERROR
...Ran 6 tests...
FAILED (failures=1, errors=4)
>>> debug == [
... # Note that the following 'suppressing ...' messages are lies
... # because no errors are effectively suppressed here.
...
... 'raising AssertionError',
... 'suppressing AssertionError',
...
... 'raising KeyError',
... 'replacing KeyError with RuntimeError',
... 'suppressing RuntimeError',
...
... 'raising OSError',
... 'replacing OSError with ZeroDivisionError',
... 'replacing ZeroDivisionError with ValueError',
... 'suppressing ValueError',
...
... 'raising UnboundLocalError',
... 'suppressing UnboundLocalError',
... 'replacing UnboundLocalError with ZeroDivisionError',
... 'suppressing ZeroDivisionError',
... 'replacing ZeroDivisionError with ValueError',
... 'suppressing ValueError',
...
... 'no error',
...
... 'no error',
... 'raising TypeError',
... 'replacing TypeError with IndexError',
... 'suppressing IndexError',
... ]
True
Note that _enable_exc_suppress_=True changes nothing when context
manager’s __exit__() returns a false value:
>>> @expand
... class AnotherSillyExcTest3(unittest.TestCase):
...
... @foreach([
... param(test_error=AssertionError)
... .context(NamedTemporaryFile, 'w+t', _enable_exc_suppress_=True),
... param(test_error=KeyError)
... .context(NamedTemporaryFile, 'w+t', _enable_exc_suppress_=True)
... .context(ErrorCM, error=RuntimeError),
... param(test_error=None)
... .context(NamedTemporaryFile, 'w+t', _enable_exc_suppress_=True),
... param(test_error=None)
... .context(NamedTemporaryFile, 'w+t', _enable_exc_suppress_=True)
... .context(ErrorCM, error=IndexError, _enable_exc_suppress_=True)
... .context(ErrorCM, error=TypeError, _enable_exc_suppress_=True),
... param(test_error=OSError)
... .context(NamedTemporaryFile, 'w+t', _enable_exc_suppress_=True)
... .context(ErrorCM, error=ValueError)
... .context(ErrorCM, error=ZeroDivisionError),
... param(test_error=UnboundLocalError)
... .context(NamedTemporaryFile, 'w+t', _enable_exc_suppress_=True)
... .context(ErrorCM, error=ValueError)
... .context(NamedTemporaryFile, 'w+t', _enable_exc_suppress_=True)
... .context(ErrorCM, error=ZeroDivisionError)
... .context(NamedTemporaryFile, 'w+t', _enable_exc_suppress_=True),
... ])
... def test_it(self, test_error):
... if test_error is None:
... debug.append('no error')
... else:
... debug.append('raising {}'.format(test_error.__name__))
... raise test_error('ha!')
...
>>> debug = []
>>> run_tests(AnotherSillyExcTest3) # doctest: +ELLIPSIS
test_it__... FAIL
test_it__... ERROR
test_it__... ERROR
test_it__... ERROR
test_it__... ok
test_it__... ERROR
...Ran 6 tests...
FAILED (failures=1, errors=4)
>>> debug == [
... 'raising AssertionError',
...
... 'raising KeyError',
... 'replacing KeyError with RuntimeError',
...
... 'raising OSError',
... 'replacing OSError with ZeroDivisionError',
... 'replacing ZeroDivisionError with ValueError',
...
... 'raising UnboundLocalError',
... 'replacing UnboundLocalError with ZeroDivisionError',
... 'replacing ZeroDivisionError with ValueError',
...
... 'no error',
...
... 'no error',
... 'raising TypeError',
... 'replacing TypeError with IndexError',
... ]
True
Context order¶
As you can see in earlier examples, a test method call can be surrouned by multiple context managers – they form a hierarchy, i.e., are nested one in another.
>>> @expand
... class DummyTest(unittest.TestCase):
... @foreach([
... param(42).context(ErrDebugCM, tag='outer')
... .context(ErrDebugCM, tag='mid-outer')
... .context(ErrDebugCM, tag='mid-inner')
... .context(ErrDebugCM, tag='inner'),
... ])
... def test(self, n):
... debug.append('test # context_targets={!r}'
... .format(current.context_targets))
...
>>> debug = [] # see earlier definition of ErrDebugCM()...
>>> run_tests(DummyTest) # doctest: +ELLIPSIS
test... ok
...Ran 1 test...
OK
>>> debug == [
... "init:outer",
... "enter:outer",
... "init:mid-outer",
... "enter:mid-outer",
... "init:mid-inner",
... "enter:mid-inner",
... "init:inner",
... "enter:inner",
... "test # context_targets=['outer', 'mid-outer', 'mid-inner', 'inner']",
... "exit:inner",
... "exit:mid-inner",
... "exit:mid-outer",
... "exit:outer",
... ]
True
As you can see, the rule seems to be simple: the higher (or more to the left) a context is declared, the more outside it is. The lower (or more to the right) – the more inside. (Let’s name this rule: “higher=outer, lower=inner”.)
Unfortunatelly, due to a design mistake made in early stages of
development of unittest_expander, this rule is not kept when we stack
up two or more foreach() decorators (to obtain the Cartesian
product of given parameter collections):
Warning
Here we describe a deprecated behavior… But, please, read on!
>>> @expand
... class DummyTest(unittest.TestCase):
... @foreach([
... param(1).context(ErrDebugCM, tag='inner'), # <- we'd prefer 'outer' here...
... ])
... @foreach([
... param(2).context(ErrDebugCM, tag='mid-outer')
... .context(ErrDebugCM, tag='mid-inner'),
... ])
... @foreach([
... param(3).context(ErrDebugCM, tag='outer'), # <- we'd prefer 'inner' here...
... ])
... def test(self, *args):
... debug.append('test # context_targets={!r}'
... .format(current.context_targets))
...
>>> debug = []
>>> run_tests(DummyTest) # doctest: +ELLIPSIS
test... ok
...Ran 1 test...
OK
>>> debug == [
... "init:outer",
... "enter:outer",
... "init:mid-outer",
... "enter:mid-outer",
... "init:mid-inner",
... "enter:mid-inner",
... "init:inner",
... "enter:inner",
... "test # context_targets=['outer', 'mid-outer', 'mid-inner', 'inner']",
... "exit:inner",
... "exit:mid-inner",
... "exit:mid-outer",
... "exit:outer",
... ]
True
What a mess!
We can, however, make the context ordering be consistently compliant
with the aforementioned “higher=outer, lower=inner” rule by setting the
expand.legacy_context_ordering global swich to False:
>>> expand.legacy_context_ordering = False
>>> @expand
... class DummyTest(unittest.TestCase):
... @foreach([
... param(1).context(ErrDebugCM, tag='outer'),
... ])
... @foreach([
... param(2).context(ErrDebugCM, tag='mid-outer')
... .context(ErrDebugCM, tag='mid-inner'),
... ])
... @foreach([
... param(3).context(ErrDebugCM, tag='inner'),
... ])
... def test(self, *args):
... debug.append('test # context_targets={!r}'
... .format(current.context_targets))
...
>>> debug = []
>>> run_tests(DummyTest) # doctest: +ELLIPSIS
test... ok
...Ran 1 test...
OK
>>> debug == [
... "init:outer",
... "enter:outer",
... "init:mid-outer",
... "enter:mid-outer",
... "init:mid-inner",
... "enter:mid-inner",
... "init:inner",
... "enter:inner",
... "test # context_targets=['outer', 'mid-outer', 'mid-inner', 'inner']",
... "exit:inner",
... "exit:mid-inner",
... "exit:mid-outer",
... "exit:outer",
... ]
True
Now the behavior is consistent and always compliant with the “higher=outer, lower=inner” rule.
And, also, is forward compatible!
Warning
In the current version of the library, the legacy (messy)
behavior described earlier is enabled when the global option
expand.legacy_context_ordering is set to True. Note
that – for backward compatibility reasons – it is by default
(however, in such a case, a deprecation warning is emitted when it
is detected that two or more foreach() decorators that bring
contexts are stacked up…). As stated above, you can swich to the
new behavior (consistently providing the “higher=outer, lower=inner”
context ordering) by setting expand.legacy_context_ordering
to False (then the deprecation warning will not be emitted).
Note that the new behavior will become the only one in the 0.6.0
version of unittest_expander; then, the only legal value of
expand.legacy_context_ordering will be False.
What to do now? To ensure your code is forward compatible, switch
to the new behavior by setting expand.legacy_context_ordering
to False and appropriately adjusting your test code (if
necessary).
Deprecated feature: accepting label and context_targets as keyword arguments¶
Warning
Here we describe a deprecated feature: automatic passing label
and/or context_targets to test methods if the expand()
decorator’s machinery detected that the method is able to accept
label and/or context_targets as keyword argument(s).
In the current version of the library, this feature is enabled when
the global option expand.legacy_signature_introspection is
set to True. Note that – for backward compatibility reasons
– it is by default (however, a deprecation warning is emitted when
it is detected that your code really makes use of the feature). The
feature can be disabled by setting
expand.legacy_signature_introspection to False (then
the deprecation warning will not be emitted).
Note that the feature will be removed from unittest_expander
in the version 0.6.0; then, the only legal value of
expand.legacy_signature_introspection will be False.
What to do now? To ensure your code is forward compatible,
make use of current.label and
current.context_targets whenever you need that information in
your test method definitions (instead of having your test methods
accepting label and/or context_targets as keyword arguments),
and switch expand.legacy_signature_introspection to
False (and, obviously, make sure your tests work correctly
with that).
If the expand.legacy_signature_introspection global option is
True and a test method is able to accept the label keyword
argument, the appropriate label (either auto-generated from parameter
values or explicitly specified with param.label()) will be passed
to the method as that argument:
>>> expand.legacy_signature_introspection = True
>>> @expand
... class Test_is_even(unittest.TestCase):
...
... @foreach(
... param(sys.maxsize, expected=False).label('sys.maxsize'),
... param(-sys.maxsize, expected=False).label('-sys.maxsize'),
... )
... def test(self, n, expected, label):
... actual = is_even(n)
... self.assertTrue(isinstance(actual, bool))
... self.assertEqual(actual, expected)
... assert label in ('sys.maxsize', '-sys.maxsize')
... sys.stdout.write(' [DEBUG: {!r}] '.format(label))
... sys.stdout.flush()
...
>>> run_tests(Test_is_even) # doctest: +ELLIPSIS
test__<-sys.maxsize> ... [DEBUG: '-sys.maxsize'] ok
test__<sys.maxsize> ... [DEBUG: 'sys.maxsize'] ok
...Ran 2 tests...
OK
Similarly, if the aforementioned option is True and a test method
is able to accept the context_targets keyword argument then a list of
context manager as-targets* (i.e., objects returned by context managers’
__enter__()) will be passed to the method as that argument:
>>> @expand
... class TestSaveLoad(unittest.TestCase):
...
... params_with_contexts = [
... (
... param(save='', load='', expected_tag='FOO')
... .context(NamedTemporaryFile, 'w+t')
... .context(debug_cm, tag='FOO')
... ),
... (
... param(save='abc', load='abc', expected_tag='BAR')
... .context(NamedTemporaryFile, 'w+t')
... .context(debug_cm, tag='BAR')
... ),
... ]
...
... @foreach(params_with_contexts)
... def test_save_load(self, save, load, expected_tag, context_targets):
... file, tag = context_targets
... assert tag == expected_tag
... file.write(save)
... file.seek(0)
... load_actually = file.read()
... self.assertEqual(load_actually, load)
... debug.append('test')
...
>>> # Note: the following change of the value of the attribute
>>> # `expand.legacy_signature_introspection` does not affect
>>> # further uses of the test class defined above because the
>>> # @expand decorator has already been applied to that class.
>>> expand.legacy_signature_introspection = False
>>> debug = [] # see earlier definition of debug_cm()...
>>> run_tests(TestSaveLoad) # doctest: +ELLIPSIS
test_save_load__<expected_tag='BAR',load='abc',save='abc'> ... ok
test_save_load__<expected_tag='FOO',load='',save=''> ... ok
...Ran 2 tests...
OK
>>> debug == [
... 'enter:BAR', 'test', 'exit:BAR',
... 'enter:FOO', 'test', 'exit:FOO',
... ]
True
Substitute objects¶
One could ask: “What does the expand() decorator do with the
original methods decorated with foreach()?”
>>> @expand
... class DummyTest(unittest.TestCase):
...
... @foreach(1, 2)
... def test_it(self, x):
... pass
...
... attr = [42]
... test_it.attr = [43, 44]
They cannot be left where they are because, without parametrization,
they are not valid tests (but rather kind of test templates). For this
reason, they are always replaced (by the expand()’s machinery)
with Substitute instances:
>>> test_it = DummyTest.test_it
>>> test_it # doctest: +ELLIPSIS
<...Substitute object at 0x...>
>>> test_it.actual_object # doctest: +ELLIPSIS
<...test_it...>
>>> test_it.attr
[43, 44]
>>> test_it.attr is test_it.actual_object.attr
True
>>> (set(dir(test_it.actual_object)) - {'__call__'}
... ).issubset(dir(test_it))
True
As you see, such a Substitute instance is kind of a
non-callable proxy to the original method (preventing it from being
included by test loaders, but still keeping it available for
introspection, etc.).
Custom method name formatting¶
If you don’t like how parametrized test method names are generated – you can customize that globally by:
setting
expand.global_name_patternto aformat()-able pattern, making use of zero or more of the following replacement fields:{base_name}– the name of the original test method,{base_obj}– the original test method (technically: function) object,{label}– the test label (automatically generated or explicitly specified withparam.label()),{count}– the consecutive number (within a single application ofexpand()) of the generated parametrized test method;
(in future versions of unittest_expander more replacement fields may be made available)
and/or
- setting
expand.global_name_formatterto an instance of a custom subclass of thestring.Formatterclass from the Python standard library (or to any object whoseformat()method acts similarily tostring.Formatter.format()).
For example:
>>> expand.global_name_pattern = '{base_name}__parametrized_{count:04}'
>>>
>>> params_with_contexts = paramseq(
... param(save='', load=''),
... param(save='abc', load='abc'),
... ).context(NamedTemporaryFile, 'w+t')
>>>
>>> @expand
... class TestSaveLoad(unittest.TestCase):
... @foreach(params_with_contexts)
... @foreach(param(suffix=' '), param(suffix='XX'))
... def test_save_load(self, save, load, suffix):
... file = current.context_targets[0]
... file.write(save + suffix)
... file.seek(0)
... load_actually = file.read()
... self.assertEqual(load_actually, load + suffix)
...
>>> run_tests(TestSaveLoad) # doctest: +ELLIPSIS
test_save_load__parametrized_0001 ... ok
test_save_load__parametrized_0002 ... ok
test_save_load__parametrized_0003 ... ok
test_save_load__parametrized_0004 ... ok
...Ran 4 tests...
OK
…or, let’s say:
>>> import string
>>> class ExtremelySillyFormatter(string.Formatter):
... def format(self, format_string, *args, **kwargs):
... count = kwargs['count']
... label = kwargs['label']
... if 'abc' in label:
... result = 'test__{}__"!!! {} !!!"'.format(count, label)
... else:
... result = super(ExtremelySillyFormatter,
... self).format(format_string, *args, **kwargs)
... if count % 3 == 1:
... result = result.replace('_', '-')
... return result
...
>>> expand.global_name_formatter = ExtremelySillyFormatter()
>>>
>>> @expand
... class TestSaveLoad(unittest.TestCase):
... @foreach(params_with_contexts)
... @foreach(param(suffix=' '), param(suffix='XX'))
... def test_save_load(self, save, load, suffix):
... file = current.context_targets[0]
... file.write(save + suffix)
... file.seek(0)
... load_actually = file.read()
... self.assertEqual(load_actually, load + suffix)
...
>>> run_tests(TestSaveLoad) # doctest: +ELLIPSIS
test--4--"!!! suffix='XX', load='abc',save='abc' !!!" ... ok
test-save-load--parametrized-0001 ... ok
test__2__"!!! suffix=' ', load='abc',save='abc' !!!" ... ok
test_save_load__parametrized_0003 ... ok
...Ran 4 tests...
OK
Set those attributes to None to restore the default behavior:
>>> expand.global_name_pattern = None
>>> expand.global_name_formatter = None
Name clashes avoided automatically¶
expand() does its best to avoid name conflicts: when it detects
that a newly generated name could clash with an existing one (whether
the latter was generated recently – as an effect of the ongoing
application of expand() – or might have already existed), it
adds a suffix to the newly generated name to avoid the clash.
As shown in the following examples, the machinery of expand() is
really careful when it comes to avoiding name conflicts:
>>> def setting_attrs(attr_dict):
... def deco(cls):
... for k, v in attr_dict.items():
... setattr(cls, k, v)
... return cls
... return deco
...
>>> @expand
... @setting_attrs({
... 'test_even__<4>': 'something',
... 'test_even__<4>__2': None,
... })
... class Test_is_even(unittest.TestCase):
...
... @foreach(
... 0,
... 4,
... 0, # <- repeated parameter value
... 0, # <- repeated parameter value
... -16,
... 0, # <- repeated parameter value
... )
... def test_even(self, n):
... self.assertTrue(is_even(n))
...
>>> Function = type(lambda: None)
>>> {name: type(obj)
... for name, obj in vars(Test_is_even).items()
... if not name.startswith('_')
... } == {
... 'test_even': Substitute,
... 'test_even__<-16>': Function,
... 'test_even__<0>': Function,
... 'test_even__<0>__2': Function,
... 'test_even__<0>__3': Function,
... 'test_even__<0>__4': Function,
... 'test_even__<4>': str,
... 'test_even__<4>__2': type(None),
... 'test_even__<4>__3': Function,
... }
True
>>> run_tests(Test_is_even) # doctest: +ELLIPSIS
test_even__<-16> ... ok
test_even__<0> ... ok
test_even__<0>__2 ... ok
test_even__<0>__3 ... ok
test_even__<0>__4 ... ok
test_even__<4>__3 ... ok
...Ran 6 tests...
OK
>>> @expand
... @setting_attrs({
... 'test_even__<0>__6': False,
... 'test_even__<0>__7': object(),
... })
... class Test_is_even_2(Test_is_even):
...
... @foreach(
... 0,
... 4,
... 0, # <- repeated parameter value
... 0, # <- repeated parameter value
... -16,
... 0, # <- repeated parameter value
... )
... def test_even(self, n):
... self.assertTrue(is_even(n))
...
>>> {name: type(obj)
... for name, obj in vars(Test_is_even_2).items()
... if not name.startswith('_')
... } == {
... 'test_even': Substitute,
... 'test_even__<-16>__2': Function,
... 'test_even__<0>__10': Function,
... 'test_even__<0>__5': Function,
... 'test_even__<0>__6': bool,
... 'test_even__<0>__7': object,
... 'test_even__<0>__8': Function,
... 'test_even__<0>__9': Function,
... 'test_even__<4>__4': Function,
... }
True
>>> run_tests(Test_is_even_2) # doctest: +ELLIPSIS
test_even__<-16> ... ok
test_even__<-16>__2 ... ok
test_even__<0> ... ok
test_even__<0>__10 ... ok
test_even__<0>__2 ... ok
test_even__<0>__3 ... ok
test_even__<0>__4 ... ok
test_even__<0>__5 ... ok
test_even__<0>__8 ... ok
test_even__<0>__9 ... ok
test_even__<4>__3 ... ok
test_even__<4>__4 ... ok
...Ran 12 tests...
OK
Questions and answers about various details…¶
“Can I omit expand() and then apply it to subclasses?”¶
Yes, you can. Please consider the following example:
>>> debug = []
>>> parameters = paramseq(
... 7, 8, 9,
... ).context(debug_cm, tag='M') # see earlier definition of debug_cm()...
>>>
>>> class MyTestMixIn(object):
...
... @foreach(parameters)
... def test(self, x):
... debug.append((x, self.n))
...
>>> @expand
... class TestActual(MyTestMixIn, unittest.TestCase):
... n = 42
...
>>> @expand
... class TestYetAnother(MyTestMixIn, unittest.TestCase):
... n = 12345
...
>>> run_tests(TestActual, TestYetAnother) # doctest: +ELLIPSIS
test__<7> (...TestActual...) ... ok
test__<8> (...TestActual...) ... ok
test__<9> (...TestActual...) ... ok
test__<7> (...TestYetAnother...) ... ok
test__<8> (...TestYetAnother...) ... ok
test__<9> (...TestYetAnother...) ... ok
...Ran 6 tests...
OK
>>> inspect.isfunction(vars(MyTestMixIn)['test']) # (not touched by @expand)
True
>>> type(vars(TestActual)['test']) is Substitute # (replaced by @expand)
True
>>> type(vars(TestYetAnother)['test']) is Substitute # (replaced by @expand)
True
>>> debug == [
... 'enter:M', (7, 42), 'exit:M',
... 'enter:M', (8, 42), 'exit:M',
... 'enter:M', (9, 42), 'exit:M',
... 'enter:M', (7, 12345), 'exit:M',
... 'enter:M', (8, 12345), 'exit:M',
... 'enter:M', (9, 12345), 'exit:M',
... ]
True
Note that, most probably, you should name such mix-in or “test template”
base classes in a way that prevents the test loader you use from
including them; for the same reason, typically, it is better to avoid
making them subclasses of unittest.TestCase.
“Can I expand() a subclass of an already expand()-ed class?”¶
Yes, you can (in some past versions of unittest_expander it was broken, but now it works perfectly):
>>> debug = []
>>> parameters = paramseq(
... 1, 2, 3,
... ).context(debug_cm) # see earlier definition of debug_cm()...
>>>
>>> @expand
... class Test(unittest.TestCase):
...
... @foreach(parameters)
... def test(self, n):
... debug.append(n)
...
>>> @expand
... class TestSubclass(Test):
...
... @foreach(parameters)
... def test_another(self, n):
... debug.append(n)
...
>>> run_tests(TestSubclass) # doctest: +ELLIPSIS
test__<1> (...TestSubclass...) ... ok
test__<2> (...TestSubclass...) ... ok
test__<3> (...TestSubclass...) ... ok
test_another__<1> (...TestSubclass...) ... ok
test_another__<2> (...TestSubclass...) ... ok
test_another__<3> (...TestSubclass...) ... ok
...Ran 6 tests...
OK
>>> type(TestSubclass.test) is type(Test.test) is Substitute
True
>>> type(TestSubclass.test_another) is Substitute
True
“Do my test classes need to inherit from unittest.TestCase?”¶
No, it doesn’t matter from the point of view of the unittest_expander machinery.
>>> debug = []
>>> parameters = paramseq(
... 1, 2, 3,
... ).context(debug_cm) # see earlier definition of debug_cm()...
>>>
>>> @expand
... class Test(object): # not a unittest.TestCase subclass
...
... @foreach(parameters)
... def test(self, n):
... debug.append(n)
...
>>> # confirming that unittest_expander machinery acted properly:
>>> instance = Test()
>>> type(instance.test) is Substitute
True
>>> t1 = getattr(instance, 'test__<1>')
>>> t2 = getattr(instance, 'test__<2>')
>>> t3 = getattr(instance, 'test__<3>')
>>> t1()
>>> t2()
>>> t3()
>>> debug == [
... 'enter', 1, 'exit',
... 'enter', 2, 'exit',
... 'enter', 3, 'exit',
... ]
True
“What happens if I apply expand() when there’s no foreach()?”¶
Just nothing – the test works as if expand() was not applied at
all:
>>> @expand
... class TestIt(unittest.TestCase):
...
... def test(self):
... sys.stdout.write(' [DEBUG: OK] ')
... sys.stdout.flush()
...
>>> run_tests(TestIt) # doctest: +ELLIPSIS
test ... [DEBUG: OK] ok
...Ran 1 test...
OK
“To what objects can foreach() be applied?”¶
The foreach() decorator is designed to be applied only to
regular test methods (i.e., instance methods, not static or class
methods) – that is, technicaly, to functions being attributes of
test (or test base/mix-in) classes.
If you apply foreach() to anything but a function object, a
TypeError is raised:
>>> @foreach(1, 2, 3)
... class What:
... '''I am not a function''' # doctest: +ELLIPSIS
...
Traceback (most recent call last):
...
TypeError: ...is not a function...
>>> @foreach(1, 2, 3)
... class WhatTest(unittest.TestCase):
... '''I am not a function''' # doctest: +ELLIPSIS
...
Traceback (most recent call last):
...
TypeError: ...is not a function...
>>> @expand
... class ErroneousTest(unittest.TestCase):
... @foreach(parameters)
... @classmethod
... def test_erroneous(cls, n):
... '''I am not a function''' # doctest: +ELLIPSIS
...
Traceback (most recent call last):
...
TypeError: ...is not a function...