Customizing error message for specific exceptions in pytest

问题

I'm trying to write a pytest plugin to customize the appearance of specific exceptions - more specifically, mocking exceptions (method expected to be called was not called etc.), because there's a lot of useless noise in the traceback of those exceptions.

This is what I've got so far, which works, but is extremely hacky:

import pytest
import flexmock

@pytest.hookimpl()
def pytest_exception_interact(node, call, report):
    exc_type = call.excinfo.type

    if exc_type == flexmock.MethodCallError:
        entry = report.longrepr.reprtraceback.reprentries[-1]
        entry.style = 'short'
        entry.lines = [entry.lines[-1]]
        report.longrepr.reprtraceback.reprentries = [entry]

I think I'm doing the right thing with the hookimpl and checking the exception type with a simple if statement.

I tried replaceing report.longrepr with a simple string, which also worked, but then I lose out on formatting (colors in the terminal).

As an example of the type of output I want to shorten, here's a mock assertion failure:

=================================== FAILURES ====================================
_______________________ test_session_calls_remote_client ________________________

    def test_session_calls_remote_client():
        remote_client = mock.Mock()
        session = _make_session(remote_client)
        session.connect()
        remote_client.connect.assert_called_once_with()
        session.run_action('asdf')
>       remote_client.run_action.assert_called_once_with('asdff')

tests/unit/executor/remote_test.py:22: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/opt/python-3.6.3/lib/python3.6/unittest/mock.py:825: in assert_called_once_with
    return self.assert_called_with(*args, **kwargs)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

_mock_self = <Mock name='mock.run_action' id='139987553103944'>
args = ('asdff',), kwargs = {}, expected = (('asdff',), {})
_error_message = <function NonCallableMock.assert_called_with.<locals>._error_message at 0x7f51646269d8>
actual = call('asdf'), cause = None

    def assert_called_with(_mock_self, *args, **kwargs):
        """assert that the mock was called with the specified arguments.

            Raises an AssertionError if the args and keyword args passed in are
            different to the last call to the mock."""
        self = _mock_self
        if self.call_args is None:
            expected = self._format_mock_call_signature(args, kwargs)
            raise AssertionError('Expected call: %s\nNot called' % (expected,))

        def _error_message():
            msg = self._format_mock_failure_message(args, kwargs)
            return msg
        expected = self._call_matcher((args, kwargs))
        actual = self._call_matcher(self.call_args)
        if expected != actual:
            cause = expected if isinstance(expected, Exception) else None
>           raise AssertionError(_error_message()) from cause
E           AssertionError: Expected call: run_action('asdff')
E           Actual call: run_action('asdf')

/opt/python-3.6.3/lib/python3.6/unittest/mock.py:814: AssertionError
====================== 1 failed, 30 passed in 0.28 seconds ======================

回答1:

If your goal is to make the stacktrace easier to read, then you can use the below code block to output a custom error message. This custom error message appears at end of the stacktrace, so you don't need to scroll up:

with raises(ZeroDivisionError, message="Expecting ZeroDivisionError"):
    pass
--> Failed: Expecting ZeroDivisionError

Source: pytest's documentation. So, instead of making a plugin you can pipe the output of pytest through something like grep and filter out the useless parts of the stacktrace.

Based on what I read in the documentation you're using the correct pytest decorator and hook function(pytest_exception_interact). But, type checking for the error might be unnecessary. This section of the documentation says that "This hook is only called if an exception was raised that is not an internal exception like skip.Exception."

回答2:

I achieve a similar goal (custom error reporting from pytest) with a similar approach to the one proposed by Mikaeil -- intercepting the output, parsing it, then applying any filters that I need.

Here are the steps:

I run pytest from a script so I can control the execution context and configuration so it is consistent.
I use this to configure pytest to output errors in json format to a log file.
I also redirect stdout and stderr to another log file (to hide it).
If pytest returns a nonzero result, I parse the json file and pass the results to a 'build monitor' class, which then outputs to stdout and to HTML.

I use the same build monitor class for all of my static analysis tools, unit tests, C compiler etc... so the build process can report errors in a nice to look at (HTML and console) and uniform way.

The relevant file for pytest is here: https://github.com/wtpayne/hiai/blob/master/a3_src/h70_internal/da/check/pytest.py

There is also a pytest plugin to control coverage reporting here: https://github.com/wtpayne/hiai/blob/master/a3_src/h70_internal/da/check/pytest_da.py

And the output formatting class is here: https://github.com/wtpayne/hiai/blob/master/a3_src/h70_internal/da/monitor/console_reporter.py

来源：https://stackoverflow.com/questions/36521111/customizing-error-message-for-specific-exceptions-in-pytest

标签

python

pytest