Collecting and Reporting pytest Results

问题

I am doing some Selenium testing via pytest. The next step is to start doing some reporting. I'd like to write something that allows me to run the tests, collect the results and send out an email. So far the closest thing that I've found to this is writing out the test result to the result log and then use a plugin to check the exist status and send an email from there. This works but is slightly cumbersome and I'm hoping that there is a more elegant way to do it. While the overall pytest documentation is great, the plugin documentation is rather poor - I can't even find pytest_sessionfinish anywhere, even though it appears to work.

import pytest

class MyPlugin:
    def pytest_sessionfinish(self, exitstatus):
        if exitstatus == 0:
            #Send success email
            pass
        else: 
            #Read output.txt
            #Add output.txt to email body
            #Send email
            pass

pytest.main("--resultlog=output.txt", plugins=[MyPlugin()])

Q: What is the best way to run and collect results from pytest?

回答1:

One easy way to generate a result report is using the pytest option --junitxml when running tests. pytest will generate a test report in JUnit format.

Since JUnit is widely used, it's easy to find tools to parse the report and generate some good looking output, like HTML reports. As far as I know, there are some plugins on Jenkins that work fine for parsing JUnit reports and provide nice reports.

Visit https://pytest.org/latest/usage.html and refer 'Creating JUnitXML format files' part.

More than that, pytest provides a way to expand the JUnit XML report when you have access to the pytest object request or config:

if hasattr(request.config, "_xml"):
    request.config._xml.add_custom_property(name, value)

If in test cases, pytest provides a fixture to do that:

def test_function(record_xml_property):
    record_xml_property("key", "value")
    assert 0

This will add a custom property to the JUnit XML report.

回答2:

If you want to implement reporting, there are many easy off the shelf solutions for pytest. Below are some of them:

Excel Reporting - https://pypi.python.org/pypi/pytest-excel
HTML Reporting - https://pypi.python.org/pypi/pytest-html
Allure Reporting (I prefer this) - https://docs.qameta.io/allure/

回答3:

Install pytest-html and then run test with --html=pytest_report.html option.

回答4:

Another way to create reports is to first collect the results as a dictoinary or dataframe, and then do whatever you like with it (write a csv, etc).

If you wish to go down that path, you can use pytest-harvest for this. Simply install it, and you can directly use the pre-defined fixtures:

import pytest
import time

@pytest.mark.parametrize('p', ['world', 'self'], ids=str)
def test_foo(p):
    """
    A dummy test, parametrized so that it is executed twice
    """
    print('\n   hello, ' + p + ' !')
    time.sleep(len(p) / 10)

def test_synthesis(module_results_df):
    """
    Shows that the `module_results_df` fixture already contains what you need
    """
    # drop the 'pytest_obj' column
    module_results_df.drop('pytest_obj', axis=1, inplace=True)

    print("\n   `module_results_df` dataframe:\n")
    print(module_results_df)

Yields

>>> pytest -s -v

============================= test session starts =============================
...
collecting ... collected 3 items
test_basic.py::test_foo[world] 
   hello, world !
PASSED
test_basic.py::test_foo[self] 
   hello, self !
PASSED
test_basic.py::test_synthesis 
   `module_results_df` dataframe:

                 status  duration_ms      p
test_id                                    
test_foo[world]  passed   500.028610  world
test_foo[self]   passed   400.022745   self
PASSED

========================== 3 passed in 0.05 seconds ===========================

You can also start from the 'dict' fixture, which contains more details about setup/teardown times, and convert it to a dataframe using the provided helper methods. See documentation for details.

Finally, if you wish to also use parameters, fixtures, steps... you might wish to look at this datascience benchmark example

I'm the author by the way ;)

来源：https://stackoverflow.com/questions/29597972/collecting-and-reporting-pytest-results

标签

python

selenium-webdriver

pytest