Calling Haskell from Python
Two ways into Haskell from Python
A Python program that wants a Haskell function has two routes:
- In process: build the function into a shared library with foreign export and load it with ctypes. Fast, no process per call, and the library has to be built on the platform that loads it.
- Out of process: run the Haskell executable with subprocess and read its output. Slower per call, no build complications, and the process cannot corrupt the Python one.
This container can build the second and not the first: the shared library fails at the linker for the reason the previous lesson showed, because the GHC runtime here is a static archive. The ctypes script is therefore written out and marked as not run here, and the measured numbers below are the subprocess route.
The ctypes script
The script loads the library, starts the runtime once, declares the two argument and result types, and calls the exported symbol. The declaration is not optional: without it ctypes passes Python ints as C int and the call may still work, or not, depending on the platform.
import ctypes
lib = ctypes.CDLL("./libadder.so")
lib.hs_init(None, None)
lib.hs_exit.argtypes = []
add = lib.hs_add
add.argtypes = [ctypes.c_int32, ctypes.c_int32]
add.restype = ctypes.c_int32
print(add(20, 22))
lib.hs_exit()
Pending run: not run in this container. The library the script loads is the one ghc -shared cannot produce here, so this output is not shown. On a machine whose GHC ships -fPIC libraries the script prints 42, and the call is a direct function call with no process involved.
What is not run here
Two things are missing on this image and neither is a flaw of the code:
- The shared library, because ghc -shared fails with relocation R_X86_64_32S ... recompile with -fPIC, as the previous lesson recorded.
- A measurement of the ctypes call, because a number for a call that did not happen would be invented. The subprocess number below is real and is the comparison the script would be measured against.
The rest of the lesson is the part that runs: a Haskell program that adds two numbers from its arguments, called from Python fifty times, timed with time.perf_counter.
The call that works here, measured
module Main (main) where
import System.Environment (getArgs)
import Text.Read (readMaybe)
main :: IO ()
main = do
args <- getArgs
case map readMaybe args :: [Maybe Int] of
[Just a, Just b] -> print (a + b)
_ -> putStrLn "usage: adder A B"
The Python side runs it fifty times and divides the elapsed time by the number of calls. check=True makes a non-zero status an exception, so a broken call is not silently counted:
import subprocess
import time
runs = 50
start = time.perf_counter()
for _ in range(runs):
subprocess.run(["./adder", "20", "22"], capture_output=True, check=True)
elapsed = time.perf_counter() - start
print(f"{runs} calls in {elapsed:.3f} s")
print(f"{elapsed / runs * 1000:.2f} ms per call")
ghc -Wall -O1 -o adder adder.hs
./adder 20 22
python3 call.py
42
50 calls in 0.129 s
2.58 ms per call
Two and a half milliseconds per call is the cost of starting a process and nothing else: the addition itself is nanoseconds. For fifty calls the subprocess route is fine; for fifty thousand, the in-process call is the only one that makes sense, which is why the shared library exists. A middle way is a long-running Haskell process that reads requests from stdin, which the lesson on stdin and stdout described and a capstone later uses.
Bytes, strings and lifetime
Three rules cover the boundary, and they apply to every FFI language pair:
- Pass numbers by value and strings as bytes. A Python str is Unicode and a C char * is bytes, so the conversion is .encode() on one side and decode() on the other.
- The bytes must outlive the call. With ctypes, a c_char_p points at a Python object; if the object is garbage collected during the call the pointer is stale. Keep the object in a variable for the duration.
- A returned char * is owned by the callee or by the runtime, not by Python, so copying it with ctypes.string_at or a free on the Haskell side is the caller's decision and must be documented.
The next lesson stays on the Haskell side and parses text with the parsec library, which is the tool a program that reads a line protocol uses before any of this marshalling.
Exercises
- Change the Haskell program to read two numbers from one line of stdin and call it once for ten pairs.
- Measure the same fifty calls with ./adder started once and fed all the pairs, and compare.
- Write the ctypes declaration for a function that takes a c_char_p and returns an int32.
- Explain why the Python string has to be kept alive during a ctypes call.
- Decide for a web service whether the in-process or the subprocess route is right and say why.
- Read the linker error of the previous lesson and say what a -fPIC build fixes.
Questions
A new set of questions in a new order every time. Choose one answer for each, then check.
-
What are the two routes from Python into Haskell?
Answer
A shared library through ctypes, or an executable through subprocess
In process with ctypes, or out of process with subprocess.
-
Why is the ctypes part marked as not run here?
Answer
The shared library cannot be linked on this image because the RTS is static
ghc -shared fails on the static archives, as the previous lesson showed.
-
What does
lib.hs_init(None, None)do?Answer
Starts the Haskell runtime before any exported call
The runtime of the shared library has to be initialised by the host process.
-
What does this program print?
module Main (main) where import System.Environment (getArgs) main :: IO () main = do args <- getArgs print (length args)Run as./count a b.Answer
2
Two arguments were passed and the program name is not counted.
-
What does the measured subprocess call cost in this container?
Answer
About 2.6 milliseconds per call
Fifty calls took 0.129 s, which is 2.58 ms each.
-
What does this program print?
module Main (main) where import System.Environment (getArgs) import Text.Read (readMaybe) main :: IO () main = do args <- getArgs case map readMaybe args :: [Maybe Int] of [Just a, Just b] -> print (a * b) _ -> putStrLn "usage"Run as./mul 6 7.Answer
42
The two numbers are parsed and multiplied.
-
Why must a Python string be kept alive during a ctypes call?
Answer
The C side holds a pointer to its bytes, not a copy
A c_char_p points into the Python object, so collecting it during the call leaves a stale pointer.
-
What does this program print?
module Main (main) where main :: IO () main = print (sum [1 .. 5 :: Int])Answer
15
The sum of 1 to 5 is 15.
-
When is the in-process route worth the shared library?
Answer
When the function is called many times and a process per call is too slow
Milliseconds per process are fine for a few calls and hopeless for thousands.
-
What does
check=Truedo insubprocess.run?Answer
Raises an exception on a non-zero exit status
A failing call becomes an exception instead of a silent wrong answer.
The code on this page was built with GHC 9.10.3 and run on Alpine Linux 3.24.3 (x86_64) in a Docker container.
Article author: Arthur Isaev