Call Python Function that Accepts OpenCV Image From C++

When working on projects that involve both C++ and Python, you may encounter scenarios where you need to pass image data between the two languages. OpenCV is a popular library for image processing, and it's commonly used in both C++ and Python environments. In some cases, you might have an existing Python function that performs a specific image processing task using OpenCV, and you want to call this function from your C++ code. This tutorial explains how to call a Python function that accepts OpenCV image from C++.

Prepare environment

Before starting, make sure you have installed Python 3 development package, CMake, C++ compiler (such as g++ or Visual C++), and OpenCV using vcpkg package manager.

Code

The following C++ code snippet demonstrates how to call a Python function named process from a script named test.py.

First, it sets up the Python and initializes necessary configurations. Then, it imports the numpy module to handle NumPy arrays, sets up the system path to include the directory of the script, and imports the test module (test.py script) where the process function resides.

It reads an image using OpenCV and converts it into a NumPy array. Next, it calls the process function with the image array as an argument, and retrieves the processed image as a NumPy array. Finally, it converts the processed NumPy array back into an OpenCV image and displays it.

project/main.cpp

#define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION
#include <Python.h>
#include <numpy/arrayobject.h>
#include <filesystem>
#include <opencv2/opencv.hpp>

int main(int argc, char *argv[])
{
    std::filesystem::path script(argv[0]);

    PyConfig config;
    PyConfig_InitIsolatedConfig(&config);

    auto fullPath = std::filesystem::current_path() / script.parent_path();
    auto venv_executable = (fullPath / "venv/bin/python").wstring();
    PyConfig_SetString(&config, &config.executable, venv_executable.c_str());

    Py_InitializeFromConfig(&config);
    PyConfig_Clear(&config);
    import_array1(-1);

    PyObject *sysPath = PySys_GetObject("path");
    PyList_Insert(sysPath, 0, PyUnicode_FromString(script.parent_path().c_str()));

    PyObject *pModule = PyImport_ImportModule("test");
    PyObject *pFunc = PyObject_GetAttrString(pModule, "process");

    cv::Mat in = cv::imread(fullPath / "test.jpg");
    npy_intp dimensions[] = {in.rows, in.cols, in.channels()};
    PyObject *pyIn = PyArray_SimpleNewFromData(in.dims + 1, dimensions, NPY_UINT8, in.data);

    auto *pyOut = (PyArrayObject *) (PyObject_CallFunction(pFunc, "O", pyIn));
    cv::Mat out((int) PyArray_DIM(pyOut, 0), (int) PyArray_DIM(pyOut, 1), CV_8UC3, PyArray_DATA(pyOut));

    imshow("Image", out);
    cv::waitKey(0);
    cv::destroyAllWindows();

    Py_Finalize();

    return 0;
}

Within the project directory, create a CMakeLists.txt file:

project/CMakeLists.txt

cmake_minimum_required(VERSION 3.24)
project(myapp)

set(CMAKE_CXX_STANDARD 17)

find_package(Python3 COMPONENTS Development REQUIRED)
find_package(OpenCV COMPONENTS highgui REQUIRED)

add_executable(${PROJECT_NAME} main.cpp)

set(VENV_PATH ${CMAKE_CURRENT_BINARY_DIR}/venv)
set(PYTHON_VERSION ${Python3_VERSION_MAJOR}.${Python3_VERSION_MINOR})
set(NUMPY_INCLUDE ${VENV_PATH}/lib/python${PYTHON_VERSION}/site-packages/numpy/core/include)

target_include_directories(${PROJECT_NAME} PRIVATE ${NUMPY_INCLUDE})
target_link_libraries(${PROJECT_NAME} Python3::Python opencv_highgui)

In the project directory, execute the following CMake command to generate the build scripts and configure the project:

cmake -S . -B build -DCMAKE_TOOLCHAIN_FILE=C:\vcpkg\scripts\buildsystems\vcpkg.cmake -DVCPKG_TARGET_TRIPLET=x64-windows

cmake -S . -B build -DCMAKE_TOOLCHAIN_FILE=/opt/vcpkg/scripts/buildsystems/vcpkg.cmake -DVCPKG_TARGET_TRIPLET=x64-linux-dynamic

Navigate to the build directory and set up a Python virtual environment:

cd build && python -m venv venv

Activate the virtual environment by running the following command:

venv\Scripts\activate.bat

source venv/bin/activate

Once the virtual environment is activated, proceed to install the OpenCV package using the following command:

pip install opencv-python

Execute the given CMake command to build the project:

cmake --build .

Inside the build directory, create a Python script named test.py for resizing image using OpenCV.

build/test.py

import cv2


def process(img):
    return cv2.resize(img, (640, 480))

Finally, put the test.jpg image in the build directory, and run the program as follows:

./myapp