Filtering Test Data

You can filter the documented test-data by using the directives needlist, needtable and needflow.

These filter functions and others are provided by the Sphinx extension Sphinx-Needs. Take a look to figure out what else is possible to customize your way of working with test cases.

Filterable data

Sphinx-Test-Reports adds the following data to a sphinx-need-configuration:

  • Types

    • testfile

    • testsuite

    • testcase

  • Options

    • file: Test file path.

    • suite: Test suite name.

    • case: Test case name.

    • suites: Amount of test suites found inside a test file.

    • cases: Amount of test cases found inside a test file.

    • passed: Amount of passed test cases.

    • skipped: Amount of skipped test cases.

    • errors: Amount of test cases with errors during execution.

    • failed: Amount of failed test cases.

Not all options are set for all created needs. E.g. test-file doesn’t include case, as it is not related to a single test case.

The :types: option expects the Sphinx-Needs type names listed above. These names do not contain hyphens. The user-facing directives still keep their hyphenated names: test-file, test-suite and test-case.

The filtering possibilities are really powerful, so take a look into Filtering needs to figure out how to get most out of your test data.

needtable - A table for all tests

needtable provides a feature-rich table of the needed test data.

Use it like:

.. needtable::
   :types: testfile
   :columns: id, file, suites, cases, passed

We set types to testfile to document needs-objects from this type only. Sphinx-Test-Reports also provides the testsuite and testcase need types.

With columns we can specify which data of a test-file we want to see.

Example

ID

File

Suites

Cases

Passed

CASPERJS_001

../tests/doc_test/utils/casperjs.xml

37

241

241

JSON_001

../tests/doc_test/utils/json_data.json

1

3

1

NOSE01

../tests/doc_test/utils/nose_data.xml

1

5

5

PYTEST01

../tests/doc_test/utils/pytest_data.xml

1

10

0

REPORT

/home/docs/checkouts/readthedocs.org/user_builds/sphinx-test-reports/checkouts/136/docs/../tests/doc_test/utils/pytest_sphinx_data_short.xml

1

7

4

SPHINX

../tests/doc_test/utils/pytest_sphinx_data.xml

1

1152

1146

TESTFILE_1

../tests/doc_test/utils/xml_data.xml

1

3

3

TESTFILE_2

../tests/doc_test/utils/pytest_data.xml

1

10

0

TESTFILE_3

../tests/doc_test/utils/nose_data.xml

1

5

5

TESTFILE_AUTO1

../tests/doc_test/utils/pytest_data.xml

1

10

0

TESTFILE_AUTO2

../tests/doc_test/utils/pytest_data.xml

1

10

0

needlist - A simple list

needlist provides a simple list of the filtered test-needs.

The filter possibilities are the same as for needtable and needflow.

Usage:

.. needlist::
   :types: testfile
   :filter: cases > 4

filter supports complex-filter operations by using a Python-statement. In this case, we check that the cases value is greater than 4.

Take a look into the Filter string section of the Sphinx-Needs documentations for more details and ideas how to use it.

Example

needflow - Flow charts of linked test data

needflow draws a picture of the filtered need-objects and their connections/links.

Note

This features needs the installed and configured sphinx-extension sphinxcontrib-plantuml.

Usage:

.. needflow::
   :types: testfile, testsuite, testcase
   :filter: len(links) > 0 or len(links_back) > 0

The used :filter: allows needs only, if they have an outgoing or incoming link.

Example