Looping through pixels with Cython still slow

问题

I get no speed difference here between regular python code. It says the bottleneck is the last two lines of code in the html file. Is there any way around this?

What I am trying to do is loop through pixels and add coordinates where rgb value is below 210 to a list.

from PIL import Image
import numpy as np
import time
import cython
import cv2

filename = "/home/user/PycharmProjects/Testing/files/file001.png"
image = Image.open(filename)
size = width, height = image.size
image_data = np.asarray(image)

cdef list list_text = []

@cython.boundscheck(False)
cpdef get_image_data():
    cdef int y, x
    for y in range(1683):
        for x in range(1240):
            if image_data[y, x] < 210:
                list_text.append([x, y])

回答1:

The looping isn't any problem, but appending lists to lists is very slow. To avoid this you can either allocate an array large enough for the data and shrink it afterwards (or copy the data in an array which has the exact size you need) or you can implement your function using std:vector.

In this answer I use Numba because I'm not that experienced in performant Cython coding, but a Cython implementation should be straight forward. Numba has also a limited internal representation of list and tuples, but I don't know if the same is available within Cython.

Example

import numpy as np
import numba as nb

@nb.njit()
def get_image_data_arr(image_data):
  array_text = np.empty((image_data.shape[0]*image_data.shape[1],2),dtype=np.int64)
  ii=0
  for y in range(image_data.shape[0]):
    for x in range(image_data.shape[1]):
      if image_data[y, x] < 210:
        array_text[ii,0]=x
        array_text[ii,1]=y
        ii+=1
  return array_text[:ii,:]

@nb.njit()
def get_image_data(image_data):
  list_text = []
  for y in range(image_data.shape[0]):
    for x in range(image_data.shape[1]):
      if image_data[y, x] < 210:
         #appending lists
         list_text.append([x, y])
         #appending tuples
         #list_text.append((x, y))
  return list_text

Timings

All timings are without compilation overhead (the first call to the function is excluded from the timings).

#Create some data
image_data=np.random.rand(1683*1240).reshape(1683,1240)*255
image_data=image_data.astype(np.uint8)


get_image_data (Pure Python)                   : 3.4s
get_image_data (naive Numba, appending lists)  : 1.1s
get_image_data (naive Numba, appending tuples) : 0.3s
get_image_data_arr:                            : 0.012s
np.argwhere(image_data<210)                    : 0.035s

回答2:

I would suggest using Numpy's argwhere() function as follows:

import numpy as np

# Create a starting image
im = np.arange(0,255,16).reshape(4,4)

That looks like this:

array([[  0,  16,  32,  48],
       [ 64,  80,  96, 112],
       [128, 144, 160, 176],
       [192, 208, 224, 240]])

Now find coordinates of all elements less than 210:

np.argwhere(im<210)

That looks like this:

array([[0, 0],
       [0, 1],
       [0, 2],
       [0, 3],
       [1, 0],
       [1, 1],
       [1, 2],
       [1, 3],
       [2, 0],
       [2, 1],
       [2, 2],
       [2, 3],
       [3, 0],
       [3, 1]])

回答3:

Ok so I sort of fixed it. Now I got to figure out how to save those pixel coordinates to a two dimensional array. Since if I append python style it slows the whole thing down. Any suggestions? I also do not really want to return image_data again.

Btw it is interesting that this code is 28000x faster than python! I expected a 100x speed gain, not this much.

@cython.boundscheck(False)
cpdef const unsigned char[:, :] get_image_data(const unsigned char[:, :] image_data):

cdef int x, y
cdef list list_text = []

for y in range(1683):
    for x in range(1240):
        if image_data[y, x] < 210:
            pass
return image_data

来源：https://stackoverflow.com/questions/53716543/looping-through-pixels-with-cython-still-slow

标签

python

numpy

cython