Combining Python and Go: Utilizing Dynamic Link Libraries to Enhance Development and Computation Efficiency

In today's software development landscape, both Python and Go are highly popular programming languages, each possessing unique advantages. Python is renowned for its concise and elegant syntax as well as its rich libraries, which significantly boost development efficiency. It is widely applied in fields such as data science, artificial intelligence, and web scraping. On the other hand, the Go language is a statically typed and compiled language. It boasts excellent concurrent performance and high running efficiency, and is often used for building high-performance network services, distributed systems, and system-level programming scenarios.

When undertaking the development of some complex projects, we often hope to fully leverage the strengths of these two languages. For example, we can develop the complex calculation part using Go, compile it into a dynamic link library, and then call it in Python. In this way, we can take advantage of Python's development convenience and also harness Go's efficient computing capabilities, thereby achieving the goal of improving development efficiency and computational speed.

Introduction to Dynamic Link Libraries

A Dynamic-Link Library (DLL for short) file is a non-executable binary program file. Its main function is to enable running programs to share the code and resources necessary for executing specific tasks. Dynamic link libraries have different file extensions in different operating systems. In the Windows operating system, dynamic link libraries often end with the .dll extension; while in the Linux system, dynamic link libraries typically end with the .so extension.

The principle of dynamic linking is to create DLL files from some commonly shared code. When an executable file calls a function in a DLL file, the Windows operating system will load the DLL file into memory. The structure of a DLL file itself is an executable file, and the functions are linked only when the program has a need for them. Through this dynamic linking method, the situation of memory waste can be greatly reduced. In contrast, static linking directly links the required code into the executable file. The original design purpose of DLLs was to save the disk and memory space required by application programs.

Go Code Example

Here we present a simple Go code example. Save it as the add.go file:

package main

import "C"  // The C library must be imported

import "fmt"

// Add the following commented code, indicating that after export, it can be called by Python
// Note that the following comment should not be written as "// export xxx", but as "//export xxx"
// Otherwise, it cannot be called

//export PrintDll
func PrintDll() {
    fmt.Println("I come from the GO dynamic link library")
}

//export Add
func Add(a int, b int) int {
    return a + b
}

func main() {
    // A main function must be added as the entry point for CGO compilation, with no specific implementation code
}

In this code, we have defined two functions, PrintDll and Add, and marked these functions as callable from Python through the special comment //export. At the same time, although the main function has no specific implementation code, it is necessary as the entry point for CGO compilation.

Compiling into a Dynamic Link Library File

Since different operating systems have different environments, the methods for compiling the add.go source file also vary. Below we will introduce it by platform:

Linux and Mac

Generally, the Linux system comes with the gcc software pre-installed, and we can directly use the following command for compilation:

go build -buildmode=c-shared -o add.so add.go

The above command will compile add.go into the add.so dynamic link library file.

Windows

To compile on Windows, you need to install the MinGW software first. After the installation is complete, you can use gcc. The compilation command is as follows:

go build -buildmode=c-shared -o add.dll add.go

This command will compile add.go into the add.dll dynamic link library file.

Optimizing the Compilation Result

Under normal circumstances, the dynamic link library file obtained through the above compilation method will be relatively large, which is also an issue that the Go language is often criticized for. However, we can use the following approach to reduce the size of the compiled file to a certain extent:

go build -ldflags "-s -w" -buildmode=c-shared -o add.so add.go

Among them, -s indicates compression; -w indicates removing debugging information. In this way, the size of the dynamic link library file can be effectively reduced.

Calling the Dynamic Link Library File in Python

To call the compiled dynamic link library file in Python, we need to use the ctypes module. The specific calling method is as follows:

# Test calling code written in the GO language from Python
# First, the GO code needs to be compiled into a dynamic link library

from ctypes import cdll

lib = cdll.LoadLibrary("./add.so")  # Specify the compiled dynamic link library file here

# Call the Add function in the GO language
result = lib.Add(100, 200)

print("result ", result)

# Call the PrintDll function in the GO language
lib.PrintDll()

The above Python code can be directly run in the Jupyter notebook. You can also save it as the add.py file and then execute it in the command line:

time python add.py

The result of running is as follows:

result 300
I come from the GO dynamic link library
python add.py  0.02s user 0.01s system 20% cpu 0.030 total

Through the above steps, we have successfully achieved calling the dynamic link library written in the Go language in Python, fully leveraging the advantages of both languages and enhancing development efficiency and computational speed.