Writing (good) unit-tests is incredibly important, but it's not always easy to understoof the correct way to test objects and methods. especially when they rely on network or filesystem objects that may not be present or feasible to test on.

This is where the idea of mocking came from, and in Python the concept is quite prevalent in many unit test cases. Let's deep dive into some demonstrations, throughout these we will shift between generic mock examples and those specific to the pytest framework which is the most popular and flexible (as comopared to the standard python unittest/pytest framework, there's also nosetest, but you can look those up).

Understanding how mocks work can be confusing at first, as it does something quite powerful and is wrapped under a layer that's very easy to use.

Example 1: Decorating

Here's a simple example that asserts the current directory is '/var', and that the os.getcwd() function is called exactly once. We replace os.getcwd with a stub, and have a MagicMock object returned as mock_getcwd by the decorator, and we can query the number of times it was called. When the code that is under mock is calling to os.getcwd(), our value is returned instead.

import os
from mock import patch

# Simple way, declare it in the decorator - short and sweet
@patch('os.getcwd', return_value='/var')
def test_path_simple(mock_getcwd):
  result = os.getcwd()
  assert mock_getcwd.call_count == 1
  assert result== "/var"

As you see, decorating @patch('os.getcwd', return_value='/var') above the method describes the fully qualified name of the method we wish to override, and we conveniently sub in the return value we want in the constructor (NB if you are testing methods in other packages, as you are likely to, the path may be different, and you may need to patch some_file.getcwd if from os import getcwd was used in some_file.py, etc).

Example 2: ContextManager

Using the patch decorator before the test method is great, but sometimes you require something larger, or more flexible. It is possible to construct and apply the mocked context at will inside a test method. Keep in mind that mock actually replaces the methods, and if used on a special or builtin function can run amok; however using context managers can handle that for us automatically, here's how.

import os
from mock import patch

def test_path_contents():
    with patch('os.getcwd', return_value="/var") as mock_getcwd:
        result = os.getcwd()
        assert mock_getcwd.call_count == 1
        assert result== "/var"

        with patch('os.listdir', return_value=['database.db']) as mock_listdir:
            result = os.listdir()
            assert mock_listdir.call_count == 1
            assert result == ['database.db']

Using the with keyword and the contextmanager, it will automatically resolve apply and unapply the mock to code in the scope of the with. In this manner multiple mocks can be applied, and they can either be done sequentially, or in combination as above; it's also possible to provide them both at once to the with command.

Example 3: Fixtures

If you are using pytest, you might benefit from reducing repetition and manual setup/teardown of mocks in test cases by applying either module or function specific fixtures. Check out the below example of using fixtures to achieve a similar result to Example 2.

import os
from mock import patch
from pytest import fixture


@fixture(scope="function", name='mocked_getcwd_fixture')
def function_fixture_getcwd(request):
    patched = patch('os.getcwd', return_value="/var")
    request.addfinalizer(lambda: patched.__exit__())
    return patched.__enter__()

@fixture(scope="function", name='mocked_listdir_fixture')
def function_fixture_listdir(request):
    patched = patch('os.listdir', return_value=['database.db'])
    request.addfinalizer(lambda: patched.__exit__())
    return patched.__enter__()

def test_with_fixtures(mocked_getcwd_fixture, mocked_listdir_fixture):
    assert os.getcwd() == "/var"
    assert os.listdir() == ['database.db']
    assert mocked_getcwd_fixture.call_count == 1
    assert mocked_listdir_fixture.call_count == 1

Now you might be asking yourself why bother when it uses an extra construct AND takes more lines of code. All reasonable questions; even though this example does much the same thing, we can see the benefit if we add a another test case. If you notice that we've defined the fixtures as functions of their own that generate something - in this case it is the Mock object for the command we wish to replace, and it also calls __enter__(), the result of that is the same as the result of calling with ... in Example 2.

def test_with_fixtures_2(mocked_getcwd_fixture, mocked_listdir_fixture):
    pass. # ...

def test_with_fixtures_3(mocked_getcwd_fixture, mocked_listdir_fixture):
    pass. # ...

We have just managed to re-use the same mock fixtures without writing them again or having the chance for errors from copy and paste. Conveniently, we can select which fixtures to use inside a test by including an argument with that name, and the fixtures plugin will detect it, creating a flexible way to mix and match depending upon the requirements of the test. the name of the argument is specified in the name keyword argument of the fixture decorator, and this is a more recent addition, as previously it could only be the default (as if unspecified now) which is the same name as the method - not always convenient. If on the other hand you always wish to use it, there's an autouse=True argument - as well as others that you can peruse.

Example 4: A more pratical use-case

Now that we have some elementary examples out of the way, let's combine some of what we used previously and try mocking something like an API client or urlrequest. In order to unit-test we need to ensure a consistent result that is predictable, and doesn't take time to load over the internet. Even if it was practical to test on a real web-service, we still need to coerce the specific responses to conduct a test, and that's where mocking comes in.

First off let's imagine a simple client for a fictional API that returns profile objects with a name and colour property. We will simplify it by having the data returned straight up or a 404 (or other) error as appropriate.

import json
import urllib.request

class SimpleClient:
    def get_profile(self, *, name: str) -> dict:
        with closing(urllib.request.urlopen(f'https://endpoint.com/{name}')) as response:
            data = response.read()
            if data is not None:
                 return (json.loads(data.decode('utf-8')), True)
            else:
                return (None, False)

That's easy enough, right? Let's write some unit tests. You might think to patch urllib.request.urlopen and that's correct, but oh wait... it requires a call to read(), and must also support close if used by a context manager. That's ok, because urllib.request.open returns an object of type http.client.HTTPResponse, and read() is called on that... so let's replace that object as the return_value, for the sake of simplicity I am calling it FakeResponse.

class FakeResponse:
    status: int
    data: bytes

    def __init__(self, *, data: bytes):
    self.data = data

    def read(self):
        self.status = 200 if self.data is not None else 404
        return self.data

    def close(self):
        pass

Simple enough right, you've even got a convenience initialiser with data= as well as calling fake_response.bytes=b''. That's great, we now have everything we need, let's write two unittests, the happy path and the unhappy path - we shall take the bad news first and write a 404 test.

from mock import patch

def test_request_404():
    with patch('urllib.request.urlopen', return_value=FakeResponse(data=None)) as mock_urlopen:
        client = SimpleClient()
        data, success = client.get_profile(name="kitten99")
        assert mock_urlopen.call_count == 1
        assert success == False
        assert data is None

def test_request_found():
    with patch('urllib.request.urlopen', return_value=FakeResponse(data=b'{"name": "Kitten", "colour": "White"}')) as mock_urlopen:
        client = SimpleClient()
        data, success = client.get_profile(name="kitten01")
        assert mock_urlopen.call_count == 1
        assert success == True
        assert data is not None
        assert data['name'] == 'Kitten'
        assert data['colour'] == 'White'

Run them, and you'll see they pass; more importantly however, you'll see not a single network call is made, we're running in imaginary land where requests are mocked and you can generate any kind of response you like to stress your object out. You'll also notice that there's a status object, as thats also part of a http.client.HTTPResponse. This class is pulled from a working project (except I used an array of requests that got popped each time to simulate sequential calls, and it used a file for some of the request data too). Here's how that would work in the FakeResponse class.

from os.path import exists
def read(self):
    self.status = 200 if exists(self.paths[0]) else 404
    with open(self.paths.pop(0), 'r') as content_file:
        return content_file.read().encode()

And that's it. Hopefully this is either a quickstart or refresh into pytest/mock frameworks.