Tests of the Python layer with unittest
The Python layer has three things that can break independently: the executable can change its answers, the mapping from an answer to a status can be wrong, and the HTTP handler can answer a request with the wrong shape. The suite below has a class for each, and the third one starts the real server on a free port and calls it over HTTP, so the test exercises the same code path a client does.
Nine tests, no test framework beyond unittest from the standard library, and no mock of the core: the first class runs the real executable, because the contract between the two languages is exactly what a mock would replace.
Three kinds of test
The executable. Four tests call run_core with a good line, a discounted line, an unknown SKU and a zero quantity, and compare the answer with the exact string the protocol promises. A change in the price list or in the message format fails here, which is the point: the line is the interface.
The mapping. Three tests call handle_line with the real runner for the happy path and the error path, and with a fake runner for the case that has no line: an answer that is neither OK nor ERR must become a 502, because that answer means the deployment is broken and the client should not read it as a bad request.
The service. Two tests start ThreadingHTTPServer on port 0, which asks the kernel for a free port, in a daemon thread, and post to it with urllib.request. One asserts a 200 with the protocol line in the JSON, the other a 400 with the code. The server is shut down and its socket closed in tearDownClass, which is what keeps the test run free of the resource warnings a leaked socket produces.
The test file
"""Tests of the Python layer: the mapping, the executable and the service.""" import json import threading import unittest import urllib.error import urllib.request from http.server import ThreadingHTTPServer from protocol import handle_line, run_core from service import Handler class RunCoreTest(unittest.TestCase): """The executable answers the protocol; one process per call.""" def test_price(self) -> None: self.assertEqual(run_core("PRICE tea 3"), "OK 750 7.50") def test_discount_above_twenty(self) -> None: self.assertEqual(run_core("PRICE coffee 3"), "OK 2632 26.32") def test_unknown_sku(self) -> None: self.assertEqual(run_core("PRICE gold 1"), "ERR E_SKU unknown sku: gold") def test_quantity_zero(self) -> None: self.assertEqual(run_core("PRICE tea 0"), "ERR E_QTY quantity out of range: 0") class HandleLineTest(unittest.TestCase): """The mapping from an answer line to a status and a payload.""" def test_success_is_200(self) -> None: status, payload = handle_line("PRICE tea 3") self.assertEqual(status, 200) self.assertEqual(payload["total"], 750) self.assertEqual(payload["money"], "7.50") def test_client_error_is_400(self) -> None: status, payload = handle_line("PRICE tea 0") self.assertEqual(status, 400) self.assertEqual(payload["error"], "E_QTY") def test_unknown_answer_is_502(self) -> None: status, payload = handle_line("PRICE tea 3", runner=lambda line: "WAT") self.assertEqual(status, 502) self.assertEqual(payload["error"], "E_CORE") class ServiceTest(unittest.TestCase): """The real HTTP server, in a thread, calling the real executable.""" server: ThreadingHTTPServer @classmethod def setUpClass(cls) -> None: cls.server = ThreadingHTTPServer(("127.0.0.1", 0), Handler) port = cls.server.server_address[1] cls.url = f"http://127.0.0.1:{port}/quote" threading.Thread(target=cls.server.serve_forever, daemon=True).start() @classmethod def tearDownClass(cls) -> None: cls.server.shutdown() cls.server.server_close() def post(self, body: str) -> tuple[int, dict]: request = urllib.request.Request(self.url, data=body.encode(), method="POST") try: with urllib.request.urlopen(request) as answer: return answer.status, json.loads(answer.read()) except urllib.error.HTTPError as error: with error: return error.code, json.loads(error.read()) def test_price_over_http(self) -> None: status, payload = self.post("PRICE tea 3") self.assertEqual(status, 200) self.assertEqual(payload["line"], "OK 750 7.50") def test_error_over_http(self) -> None: status, payload = self.post("PRICE gold 1") self.assertEqual(status, 400) self.assertEqual(payload["error"], "E_SKU") if __name__ == "__main__": unittest.main()
The ServiceTest class is the one worth reading twice. A server on port 0 is a server the test does not have to coordinate about, a daemon thread dies with the process even when the shutdown fails, and the post helper turns the 400 into a value instead of an exception so that both outcomes are one line in the test.
The run
Verbose mode prints one line per test. The two lines in the middle that look like HTTP are the handler's log, which the server writes to standard error while the integration tests run:
python3 -m unittest -v test_protocol.py; echo "status $?"
test_client_error_is_400 (test_protocol.HandleLineTest.test_client_error_is_400) ... ok
test_success_is_200 (test_protocol.HandleLineTest.test_success_is_200) ... ok
test_unknown_answer_is_502 (test_protocol.HandleLineTest.test_unknown_answer_is_502) ... ok
test_discount_above_twenty (test_protocol.RunCoreTest.test_discount_above_twenty) ... ok
test_price (test_protocol.RunCoreTest.test_price) ... ok
test_quantity_zero (test_protocol.RunCoreTest.test_quantity_zero) ... ok
test_unknown_sku (test_protocol.RunCoreTest.test_unknown_sku) ... ok
test_error_over_http (test_protocol.ServiceTest.test_error_over_http) ... "POST /quote HTTP/1.1" 400 -
ok
test_price_over_http (test_protocol.ServiceTest.test_price_over_http) ... "POST /quote HTTP/1.1" 200 -
ok
----------------------------------------------------------------------
Ran 9 tests in 0.555s
OK
status 0
Nine tests in about half a second, and the half second is six process starts, four in the first class and two behind the HTTP tests. The suite is cheap enough to run on every edit, which is the only property that decides whether a test suite is used.
What each class pins down
| Class | What it calls | What breaks it |
|---|---|---|
| RunCoreTest | the real executable, one process per call | a price change, a message change, a code change in the core |
| HandleLineTest | the mapping, with a real runner and with a fake one | a wrong status for a code, a missing field in the JSON |
| ServiceTest | the real HTTP server in a thread, over urllib | a broken handler, a wrong content type, a missing path check |
The three classes form a small pyramid: the cheap mapping tests at the bottom, the process tests in the middle, the two HTTP tests on top. The numbers are not an accident. The mapping is pure and cheap, so it gets the cases that are hard to produce with a real core; the process and the socket are slow, so they get one case each of the happy path and one of the error path.
What is not tested
Three things are missing on purpose, and each one is a decision a reader can disagree with. The shared library is not tested: its only Python client is the benchmark of the timing page, and a test of it would be a test of ctypes plus the same core. Concurrency is not tested: the server is threaded and the handler starts a process, so a test with ten clients at once would be the real test of the design, and it would also be the flakiest one in the file. And the timing numbers are not asserted: a test that fails when a machine is slow is a test that fails on the build server and passes on the laptop.
What the suite does assert is the contract between the languages, and that is the part that changes. A price edit in Haskell fails a test in Python, which is the whole argument for a line protocol: the two languages do not share a type, so the only thing they can share is a test.
The next page writes the Dockerfile that ships this, and marks it as not built here, because this container has no Docker. The Haskell lesson on testing a program from outside is Capstone A: testing the parser.
The tests ran here with Python 3.14.10 and GHC 9.10.3 on Alpine Linux 3.24.3 (x86_64) in a Docker container.
Article author: Arthur Isaev