PYTHON- python 与 c++ 相互调用

url：http://answers.opencv.org/question/189772/python-c-swig-share-image-pixel-data-in-memory/

PYTHON C++ SWIG share image pixel data in memory

Hello,

I am working on passing image pixel data from c++ app to python and vice versa. The use case is user creates an image in the c++ application and then retrieve the image (preferably from memory so that we do not need to copy back and forth) and do some image process in python (using python image processing library such as PIL, opencv etc). Once image processing is done in python, we would want to pass the image back to the c++ application (preferably from memory as well, so that we do not need to copy the image data).

So, my question is how to get the image pixel data from python? I can pass the void* that points to the pixel data as well as the width and height information from the C++ application (with SWIG).

And how do we pass the image pixel data from python back to the c++ app (again prefer to just refer it to the pixel data in memory)?

My application is written in c++ and I am using SWIG to interface with python.

Thanks.

Comments

Is your C++ going into a DLL? Have you tried it just to see if it works, before you get into SWIG?

sjhalayka  (Apr 21 '18)

Yes, the c++ is a dll. In fact it is now a .pyd since I am using SWIG. I can pass the void* (image data is unsigned char in memory) to python. But I cannot seem to find a library to construct the data from memory (in terms of void* address) to create an image (given that from python, I know the rows and column of the image). I am still exploring how to do it using numpy/opencv/PILLOW.

Do you know how to do it without swig?

wlei  (Apr 21 '18)

This code sets up a new Mat from a void* pointer (called ptr):

Mat frame_content = Mat(480, 640, CV_16UC1, ptr).clone();

Does that help?

sjhalayka  (Apr 21 '18)

Thanks, but Mat is not available in cv2, right? I think I need a function in python (openCV or numpy or which ever image processing library that can create the image back from the block of memory (void * from c++).

wlei  (Apr 21 '18)

I'm not sure what OpenCV 2.x's capabilities are. Sorry about that.

sjhalayka  (Apr 21 '18)

... so you're looking to pass the image data from C++ to Python, right?

sjhalayka  (Apr 21 '18)

Thanks. Yes, both ways from C++ to python (get image from c++ app, use the image in python to do some image processing and let the C++ app pick up the latest image data) and then from python back to C++.

So, if we use the Mat mechanism, how do we expose it to python to pick it up to do processing?

wlei  (Apr 21 '18)

That's a good question, and I'm going to look into it. Do you have any source code that I can work from? I use GitHub for sharing code.

sjhalayka  (Apr 21 '18)

I am using python 3.6 for 64 bit. I haven't tried to build c++ dll with OpenCV yet. Actually, right now, my cpp code does not use opencv yet. I am still finding way to expose the data to python. Looks like there is a python buffer protocol that I can make use of... I still need to explore it... have you use it?

So far, I am using opencv on the python side to try to pick up the data from memory address... no luck so far...

sorry, my source code is linking all over the places...too much dependencies that will require a lot of time to set up... thanks though!

wlei  (Apr 21 '18)

Do you have a working example of a void** parameter function?

sjhalayka  (Apr 22 '18)

see more comments

2 answers

Sort by » oldest newest most voted

1

answered Apr 23 '18

sjhalayka
571 ●4 ●8

updated Apr 23 '18

I exported a DLL function that returns a char* array. The Python code calls the function and converts the bytes into a 2D numpy array, and back again to bytes:

Input image:

Windows code:

#include <opencv2/opencv.hpp>
using namespace cv;
#pragma comment(lib, "opencv_world331.lib")

#include <iostream>
#include <iomanip>
#include <string>
#include <map>
#include <mutex>
using namespace std;


#define DLLEXPORT extern "C" __declspec(dllexport)


map<size_t, char*> ids;
mutex rw_ids;

DLLEXPORT int get_img_from_char_array(char *c, int *width, int *height)
{
    Mat frame_content(*height, *width, CV_8UC1, c);

    return 0;
}


DLLEXPORT char *alloc_img(const char *file_name, int *width, int *height, size_t *image_id)
{
    rw_ids.lock();

    Mat img = imread(file_name, IMREAD_GRAYSCALE);

    if (img.empty())
    {
        *width = 0;
        *height = 0;

        rw_ids.unlock();

        return nullptr;
    }

    *width = img.cols;
    *height = img.rows;

    size_t num_pixels = (*width)*(*height);

    char *p = new char[num_pixels];
    memcpy(p, img.data, num_pixels*sizeof(char));

    ids[ids.size()] = p;
    *image_id = ids.size() - 1;

    cout << "Created img id " << *image_id << endl;

    rw_ids.unlock();

    return p;
}

DLLEXPORT int free_img(size_t *image_id)
{
    rw_ids.lock();

    for (map<size_t, char*>::const_iterator ci = ids.begin(); ci != ids.end(); ci++)
    {
        if (ci->first == *image_id)
        {
            cout << "Deleted img id " << *image_id << endl;

            if (nullptr != ci->second)
                delete[] ci->second;

            ids.erase(ci);

            rw_ids.unlock();

            return 0;
        }
    }

    rw_ids.unlock();

    return 1;
}

Python code:

import ctypes as ct
import numpy as np
import cv2

ptr = ct.c_char_p()
s = "dove.png"
width = ct.pointer(ct.c_int())
height = ct.pointer(ct.c_int())
img_id = ct.pointer(ct.c_size_t())
lib = ct.CDLL("void_dll.dll")
alloc_img = lib.alloc_img
free_img = lib.free_img
get_img_from_char_array = lib.get_img_from_char_array

alloc_img.restype = ct.c_char_p

bytes_array = alloc_img(ct.c_char_p(s.encode("utf-8")), width, height, img_id)
numpy_array = np.frombuffer(bytes_array, dtype=np.uint8)
numpy_array.shape = (width.contents.value, height.contents.value)

cppbytes = numpy_array.tobytes()
get_img_from_char_array(cppbytes, width, height);

free_img(img_id)

I also have some extra code handy, which uses a void** pointer to allocate memory:

int func(void **ptr)
{
    float *p = new float[50];
    p[0] = 123.456;
    *ptr = p;

    return 0;
}

int main(void)
{
    float *p = 0;
    func(reinterpret_cast<void**>(&p));

    cout << p[0] << endl;

    delete[] p;

    return 0;
}

link

Comments

1

Thanks for the detailed information.

I was able to get it working with Python buffer and memory view from the C++ side. In case you are interested. Here is the code:

PyObject* :GetBufferPtr(Image img, long width, long height){..... Py_buffer pybuffer; //where buf points to a void* for image pixel int res = PyBuffer_FillInfo(&pybuffer, 0, buf, val, false, PyBUF_CONTIG); .. return PyMemoryView_FromBuffer(&pybuffer);}

On the Python side, I was able to retrieve the image pixel using numpy.frombuffer() function:np_array = np.frombuffer(imgPtr, dtype=np.uint8)np_array.shape = (rows, cols) //where rows and cols are the width and height of the image

The above function can be used with or without SWIG...

wlei  (Apr 24 '18)

Right on. Glad to hear you got it working.

If you like my answer, then please mark it as correct.

sjhalayka  (Apr 24 '18)

Thank you.

Now, the other part is how to write to the shared memory from python side. I am able retrieve the numpy array and creates/displays the image. Now, if I do some processing on the image (say flip the image 90 degree) and I want to save the new image data to the share memory, how do I do that in python?

sorry, I am pretty new to this forum, I will have to figure out how to mark the answer as correct...

wlei  (Apr 24 '18)

I also put my code up at https://github.com/sjhalayka/python_c...

In the C++ file void_dll.cpp, there is a function get_img_from_char_array() that takes a Python byte array and converts it into a Mat object. The void_dll.py file calls the function from the DLL.

Perhaps this may be inspirational to you.

sjhalayka  (Apr 24 '18)

There is a check mark on the left of the top of my answer. There are also an up arrow and down arrow, so you could vote me up that way too, if you like.

And thank you for the inspiration to write some code.

sjhalayka  (Apr 25 '18)

Which version of OpenCV are you using?

sjhalayka  (Apr 25 '18)

I am using the latest opencv (3.4.1)

The get_img_from_char_array does not help as I was looking at the code on python side to update the memory directly (that was shared by the c++ app) so that the c++ app just need to do a refresh to pick up the change from the python (i.e. c++ app should not need any new code to achieve this as only the memory content is changed so it should still be able to update data pointed by the void* that was shared with the python buffer).

I was able to update the buffer on the python side and display the image using the latest change in the buffer. But, for some unknown reason, when c++ pick up the change, only part of the memory is updated.

Here is the code on the python side to update the buffer (pls. see next comment):

wlei  (Apr 25 '18)

import myLib import cv2 import numpy as np imgPtr = myLib.GetImgBuffer(img1, cols1, rows1) numpy_array = np.frombuffer(imgPtr, dtype=np.uint8) numpy_array.shape = (rows1, cols1)

//assume: rows1 = cols1 = 1024
//change the image in python
dst = np.ones((1024,1024), dtype=np.uint8)

//update data in the buffer
np.copyto(numpy_array, dst) //IS THIS CORRECT FOR UPDATING THE MEMORY IN THE BUFFER??
numpy_array.shape = dst.shape

 //create another numpy array from the updated buffer
numpy_array2 = np.frombuffer(imgPtr, dtype=np.uint8)
numpy_array2.shape = numpy_array.shape // this array display the correct change but when we pick up 
//the change in c++, only ~ quarter of the image is changed)
cv2.imshow('img - retrieve image from updated buffer & display in Python', numpy_array2)
cv2.waitKey(0)

wlei  (Apr 25 '18)

Can you please paste your code into a Gist on Github?

sjhalayka  (Apr 25 '18)

yes, it is located at https://gist.github.com/wlei2018/ff5c...

wlei  (Apr 25 '18)

see more comments

0

answered Apr 26 '18

wlei
39 ●1 ●5

updated Apr 26 '18

sjhalayka
571 ●4 ●8

To expose the shared python buffer from C++ side:

PyObject* :GetBufferPtr(Image img, long width, long height) 
{ .....
  Py_buffer pybuffer; //where buf points to a void* for image pixel 
  int res = PyBuffer_FillInfo(&pybuffer, 0, buf, val, false, PyBUF_CONTIG); 
  .. 
  return PyMemoryView_FromBuffer(&pybuffer); 
}

To retrieve image buffer from Python's side and update the buffer from Python side:

import myLib
import cv2 
import numpy as np 
imgPtr = myLib.GetImgBuffer(img1, cols1, rows1) 
numpy_array = np.frombuffer(imgPtr, dtype=np.uint8)  //for one byte integer image
numpy_array.shape = (rows1, cols1)
//change the image in python
dst = np.ones((rows1,cols1), dtype=np.uint8)

//update data in the buffer
np.copyto(numpy_array, dst) //update the buffer
numpy_array.shape = dst.shape

**Now, if you refresh your display in the C++ app, you will see the updated change from Python

Have fun coding!

link

Comments

I edited your answer to codify the code. I also rated you up!

sjhalayka  (Apr 26 '18)

So what does SWIG do???

sjhalayka  (Apr 26 '18)

In this case, we don't really need SWIG to do the interfacing since we already return a PyObject*. But, if one is already using SWIG, it will still work with the .i (interface) file.

Thanks for your rating!

wlei  (Apr 26 '18)

add a comment