Test framework integration

We have strong opinions about the testing tools we use, and we understand that other people may have equally strong opinions that differ from ours. For this reason, we have decided that none of Seaworthy’s core functionality will depend on pytest, testtools, or anything else that might get in the way of how people might wants to write their tests. On the other hand, we don’t want to reinvent a bunch of integration and helper code for all the third-party testing tools we like, so we also provide optional integration modules where it makes sense to do so.

pytest

Seaworthy is a pytest plugin and all the functions and fixtures in the seaworthy.pytest module will be available when Seaworthy is used with pytest.

docker_helper fixture

A fixture for a DockerHelper instance is defined by default.

This fixture creates DockerHelper instances with default parameters and has module-level scope. Since all other Docker resource fixtures typically depend on the docker_helper fixture, resources must have a scope smaller than or equal to the docker_helper’s scope.

The defaults for this fixture can be overridden by defining a new docker_helper fixture using the docker_helper_fixture() fixture factory. For example:

from seaworthy.pytest.fixtures import docker_helper_fixture

docker_helper = docker_helper_fixture(scope='session', namespace='seaworthy')

…would change the scope of the docker_helper fixture to the session-level and change the namespace of created Docker resources to seaworthy.

dockertest decorator

The dockertest() decorator can be used to mark tests that require Docker to run. These tests will be skipped if Docker is not available. It’s possible that some tests in your test suite may not require Docker and you may want to still be able to run your tests in an environment that does not have Docker available. The decorator can be used as follows:

@dockertest()
def test_docker_thing(cake_container):
    assert cake_container.exec_cake('variant') == ['gateau']

Fixture factories

A few functions are provided in the seaworthy.pytest.fixtures module that are factories for fixtures. The most important two are:

resource_fixture(definition, name, scope='function', dependencies=())

Create a fixture for a resource.

Note

This function returns a fixture function. It is important to keep a reference to the returned function within the scope of the tests that use the fixture.

fixture = resource_fixture(PostgreSQLContainer(), 'postgresql')

def test_container(postgresql):
    """Test something about the PostgreSQL container..."""

Parameters:

• definition – A resource definition, one of those defined in the seaworthy.definitions module.
• name – The fixture name.
• scope – The scope of the fixture.
• dependencies – A sequence of names of other pytest fixtures that this fixture depends on. These fixtures will be requested from pytest and so will be setup, but nothing is done with the actual fixture values.

Returns:

The fixture function.

clean_container_fixtures(container, name, scope='class', dependencies=())

Creates a fixture for a container that can be “cleaned”. When a code block is marked with @pytest.mark.clean_<fixture name> then the clean method will be called on the container object before it is passed as an argument to the test function.

Note

This function returns two fixture functions. It is important to keep references to the returned functions within the scope of the tests that use the fixtures.

f1, f2 = clean_container_fixtures(PostgreSQLContainer(), 'postgresql')

class TestPostgresqlContainer
    @pytest.mark.clean_postgresql
    def test_clean_container(self, web_container, postgresql):
        """
        Test something about the container that requires it to have a
        clean state (e.g. database table creation).
        """

    def test_dirty_container(self, web_container, postgresql):
        """
        Test something about the container that doesn't require it to
        have a clean state (e.g. testing something about a dependent
        container).
        """

Parameters:

• container – A “container” object that is a subclass of ContainerDefinition.
• name – The fixture name.
• scope – The scope of the fixture.
• dependencies – A sequence of names of other pytest fixtures that this fixture depends on. These fixtures will be requested from pytest and so will be setup, but nothing is done with the actual fixture values.

Returns:

A tuple of two fixture functions.

testtools

We primarily use testtools when matching against complex data structures and don’t use any of its test runner functionality. Currently, testtools matchers are only used for matching PsTree objects. See the API documentation for the seaworthy.ps module.

Testing our integrations

To make sure that none of the optional dependencies accidentally creep into the core modules (or other optional modules), we have several sets of tests that run in different environments:

• tests-core: This is a set of core tests that cover basic functionality. tox -e py36-core will run just these tests in an environment without any optional or extra dependencies installed.
• tests-pytest, etc.: These are tests for the optional pytest integration modules. tox -e py36-testtools will run just the seaworthy.pytest modules’ tests in an environment with only the necessary dependencies installed.
• tests-testtools, etc.: These are tests for the optional testtools integration module. tox -e py36-testtools will run just the seaworthy.testtools module’s tests.
• tests: These are general tests that are hard or annoying to write with only the minimal dependencies, so we don’t have any tooling restrictions here. tox -e py36-full will run these, as well as all the other test sets mentioned above, in an environment with all optional dependencies (and potentially some additional test-only dependencies) installed.