This article will show how to use the built-in “ctypes” module in Python to use external C language libraries.
1. Introduction
As developers, we can work with a lot of programming languages during our careers. Sometimes, we need to extend external libraries from another application, like a legacy system or even a shared OS library (.dll, .so, and so on…).
Python introduces the ctypes module, which works as an interface to load and handle C code. This module is powerful and came with the standard Python package.
In the next sections, I’ll show how to load a simple C library and do some basic things like calling a function and dealing with data types.
1.1 Pre-requisites
The examples found in this article work in version 3.6 or above. You can find the most recent version of Python here.
A C compiler is also needed to work with the C code examples. Linux and macOS already have the GCC compiler embedded. For Windows users, I recommend using Visual Studio C/C++ that came with the clang compiler. Maybe you’ll need some support to compile the C code that you can find easily on the internet.
Also, feel free to use any IDE/Code Editor with support for Python language and versions recommended above.
2. Simple C library used in Python
The code below is as simple as must be. Since the target of this article is talking about the ctypes module, I’ve created a simple “Hello World” to use as an example.
mylibc.c
#include <stdio.h>
const char* hello() {
input = "Hello World!";
return input;
}
To use as an external library, we’re going to compile using the following command:
Compiling C code
$ gcc -shared -o mylibc.so mylibc.c // For windows $ gcc -shared -o mylibc.dll mylibc.c
After compilation, we have the mylibc.so generated. Depending on your OS, the .so is a shared object, which means that other applications can access that library if they have an interface to do it.
On Linux/macOS systems, the .so is a standard. For Windows, we generally use .dll files as a shared library.
In Python, we’ll use the module ctypes to interface with that C library we just created previously.
3. Calling Simple functions with “ctypes” module
Now, we’re going to use the REPL Python console. Use the command below to start it on your terminal. In this article, I’m using Python 3.9.1.
Python Console
$ python3.9 Python 3.9.1 (default, Jan 30 2021, 15:51:05) [Clang 12.0.0 (clang-1200.0.32.29)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import ctypes >>>
The first move was to import the ctypes module. Just type as above to put into the console. Next, we have these commands to work with the C library.
Managing C Library
>>> libc = ctypes.CDLL("mylibc.so")
>>> c_hello_func = libc.hello
>>> c_hello_func.restype = ctypes.POINTER(ctypes.c_char)
>>> hello_func = c_hello_func()
>>> phrase = ctypes.c_char_p.from_buffer(hello_func)
>>> print(phrase.value)
b"Hello World! I'm a C library!"
Walking through the code above:
- The
ctypes.CDLL()is the module function to load the library to the Python code. - With the
libcobject created in Python, we start to handle the method of this object (in this case the hello() method). - Further, C has some tricks that we handle using
ctypes.POINTER. This function will tell to our objectc_hellothe method’s return type (c_char). - Next, we create a new object (
hello_func) to deal with the return of thec_hello_funcobject. Notice here we are working with C data yet. - Later, we use
ctypes.c_char_p.from_buffer()to encapsulate it into a final object (phrase). - Finally, we print the return using Python’s
print()method. The result is the famous “Hello World!”.
What we could see here? It’s not so tricky to load a C library into Python, but we have some work on handling data. The ctypes module has a lot of functions to make this interface easier and we can implement this integration quickly.
4. Mutable and immutable Strings
When we develop in Python, we need to know that strings are immutable. That’s to say, a string cannot change from its previous state.
Using ctypes module, we can handle this situation easily. With the example below, we’ll reverse a string using the C library.
reverselib.c
#include<stdio.h>
void reverse(char *str)
{
int len;
int tmp = 0;
for(len=0; str[len] != 0; len++);
for(int i = 0; i <len/2; i++)
{
tmp = str[i];
str[i]=str[len - 1 - i];
str[len - 1 - i] = tmp;
}
}
We can load and pass some strings to our C object in Python, but the result will not be as expected.
Using function in Python
import ctypes
def reverse_string(libc, strg):
print("String before: ", strg)
libc.reverse(strg)
print("String after: ", strg)
if __name__ == "__main__":
LIBC = ctypes.CDLL("reverselib.so")
strg = "oigres"
reverse_string(LIBC, strg)
Result with immutable string
String before: oigres String after: oigres
Now, with a little change in our reverse_string function, we convert the original string to bytes using the str.encode function and then pass it to the ctypes.string_buffer. String buffers are mutable and now we can work properly with them in Python.
Change to mutable string
def reverse_string(libc, strg):
mutable_string = ctypes.create_string_buffer(str.encode(strg))
print("String before: ", mutable_string.value)
libc.reverse(mutable_string)
print("String after: ", mutable_string.value)
Result with mutable string
String before: b'oigres' String afer: b'sergio'
5. Function signatures
Interfacing with external libraries, especially those written in other languages, requires some care from the user. In other words, our integration needs to translate to the external libraries the data type that is required to use in the library.
Of course, the ctypes module has a series of functions to help with that struggle. We need to transform our data to the type of library function signature. Also, the return type is handled as we saw in our first example.
I’ve created the example below to explain how to work with function signatures. It’s a simple BMI calculator as we can see.
bmicalclib.c
#include <stdio.h>
char* calc(double height, double weight) {
double heightSquare = height * height;
double bmi = weight / heightSquare;
char* result = "Underweight (<18.5)";
if(bmi > 18.5 && bmi < 25.0) {
result = "Normal weight (>18.5 <25.0)";
} else if(bmi > 25.0 && bmi < 30.0) {
result = "Overweight (>25.0 <30.0)";
} else if(bmi > 30.0 && bmi < 35.0) {
result = "Obese (>30.0 <35.0)";
} else if(bmi > 35.0) {
result = "Super Obese (>18.5 <25.0)";
}
return result;
}
Now, our Python code to work with the C LIbrary above.
bmi_calc.py
import ctypes
def calc(libc, height, weight):
print("Starting calculation for Height {0} and Weight {1}".format(height, weight))
c_calc_func = libc.calc
# First, treat return type
c_calc_func.restype = ctypes.POINTER(ctypes.c_char)
# Now, marshall the function attributes
c_height = ctypes.c_double(height)
c_weight = ctypes.c_double(weight)
# And create a new object to store the return
calc_func_obj = c_calc_func(c_height, c_weight)
result = ctypes.c_char_p.from_buffer(calc_func_obj)
print("Result: {0}".format(result.value))
Firstly, we put the library function into an object (c_calc_func). Moving on, we use the ctypes.restype function to use a POINTER using the type char (ctypes.c_char) to treat the function later.
Further, We can notice that ctypes convert the attributes height and weight to the C format using the function ctypes.c_double. Also, we’ve created another object to store the returning data and finally print the result.
BMI calc result
$ python3.9 -m bmi_calc Starting calculation for Height 1.7 and Weight 71.0 Result: b'Normal weight (>18.5 <25.0)'
6. Memory management basics
The C language hasn’t a garbage collector as default, so, it’s the user’s duty to guarantee free memory after the application uses it.
One advantage to working with Python is that we don’t spend time doing this manual memory management. However, sometimes we need to allocate some memory in C and pass it to Python for some manipulation. That was what we did in our previous example.
We already use some jobs allocating memory for the C library. But I created this example below to show how to free memory on C.
libmemorybasics.c
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
char * alloc_C_string(void) {
char* phrase = strdup("I was written in C");
printf("C just allocated %p(%ld): %s\n",
phrase, (long int)phrase, phrase);
return phrase;
}
void free_C_string(char* ptr) {
printf("About to free %p(%ld): %s\n",
ptr, (long int)ptr, ptr);
free(ptr);
}
The alloc_C_string function shows the address and the size memory (in bytes) allocated by C to create the variable. The free_C_string function, obviously, will free that memory using the free() function from standard C libraries.
Continuing, our Python code will work with the C library.
memory_basics.py
import ctypes
def alloc_mem(libc):
alloc_c_string = libc.alloc_C_string
alloc_c_string.restype = ctypes.POINTER(ctypes.c_char)
c_string_address = alloc_c_string()
phrase = ctypes.c_char_p.from_buffer(c_string_address)
print("Bytes in Python {0}".format(phrase.value))
free_func = libc.free_C_string
free_func.argtypes = [ctypes.POINTER(ctypes.c_char), ]
free_func(c_string_address)
if __name__ == "__main__":
LIBC = ctypes.CDLL("libmemorybasic.so")
alloc_mem(LIBC)
Explaining the code above, we used again the ctypes.POINTER function to manipulate the alloc_C_string function. Also, use the ctypes.c_char_p.from_buffer to store the return in a new object.
The new thing here is the .argtypes used in the created free_func object. Furthermore, here is another (there is a lot of) type of marshaling to handle the data type.
Basically, the free_func will take the previously allocated object from the C library and execute the function to free the memory space.
7. Summary
In conclusion, we saw how to import and use a C library in Python using the ctypes module. Further, we could handle data types using the ctypes functions, and understand the steps to implement the properly Python code to work together with the C library.
As a piece of advice, read the official Python documentation about ctypes here. It brings more information about this module and you can improve your understanding of this tool.
8. Download the source code
You can download the full source code of this example here: ctypes in Python
