Convert rows of hexadecimal values to binary, vertically by column

问题

I am working with data coming from a serial device which outputs its data in a very interesting format. The device has a 256x256 array of pixels, whereas each pixel has a 14-bit value, read-out with a shift register.

To show the format, I will illustrate it as it would look if each pixel had a 6-bit value:

                             'Pixel #'
        0-8   9-16   17-24  25-32  33-40  41-48  48-56  57-64 ... 256
        --------------------------------------------------------------
    0   255    255    255    255    255    255    255    255  ...
    1   127    255    255    255    255    255    255    255  ...
    2   255    255    255    255    255    255    255    255  ...
    3   255    255    255    255    255    255    255    255  ...
    4   255    255    255    255    255    255    255    255  ...
    5   255    255    255    255    255    255    255    255  ...

    Note that the 2nd row starts with a value of 127

To get the 6-bit value of the 1st pixel (pixel # 0), the following must occur:

Every value in row needs to be treated as its binary/bit equivalent
Reading vertically, the aligned bits from each row, 6 rows down (for a 6-bit output value), results in the value of that pixel

That is:

                             'Pixel #'
          0-8     9-16     17-24    25-32  ...  256
        --------------------------------------------------------------
    0   *1*1111111 11111111 11111111 11111111 ...
    1   *0*1111111 11111111 11111111 11111111 ...
    2   *1*1111111 11111111 11111111 11111111 ...
    3   *1*1111111 11111111 11111111 11111111 ...
    4   *1*1111111 11111111 11111111 11111111 ...
    5   *1*1111111 11111111 11111111 11111111 ...

    Note that the 2nd row had a value of 127, which is 01111111 in binary
    --> Pixel 0 = 101111 = 47

Now, repeat that across all 256 columns, then move down to the next 6 rows and repeat.

The actual output needs to be an array of pixel values that is 256x256. The actual dataset I need to process is 14-bits for each pixel - it's 3584x32 (14-bits * 256 pixels = 3584 rows ... and 32 bytes * 8 bits/byte = 32 bytes across).

What would be the best way to process the dataset? Also, speed is of critical concern, so are there some high-speed functions that could be leveraged?

Appreciate the help - Thanks!

EDIT:

To answer the questions about the required speed - Ideally, I would like to perform this at least 10x/sec, as the data is coming in at 60x/sec. Therefore, I think I'd need to avoid the common 'joins' and string operations, as I believe those to be quite slow.

Again, the real dataset (3584x32) has 14-bits for each pixel, so it's 3584x32.

Here's the function I am working on, using Joran's method, which takes ~2.6 secs to execute when provided with the real dataset:

def GetFrame(RawData):
    if np.shape(RawData) == (3584, 32):
        ProcessedData = np.zeros((256, 256), dtype='int16')
        data_blocks = [RawData[d:d+14] for d in range(0, 3584, 14)]
        for p in range(256):
            data_bin_rows = ["".join(map(lambda val:"{0:08b}".format(val,), row)) for row in data_blocks[p]]
            ProcessedData[p][:] = [int("".join(v),2) for v in zip(*data_bin_rows)]
        return ProcessedData
    else:
        return False

How can that be made faster to get the execution time down? Thanks!

回答1:

I had to read it a few times but I think Ive got it

data = \
"""255    255    255    255    255    255    255    255
127    255    255    255    255    255    255    255
255    255    255    255    255    255    255    255
255    255    255    255    255    255    255    255
255    255    255    255    255    255    255    255
255    255    255    255    255    255    255    255"""

data_rows = [map(int,row.split()) for row in data.splitlines()]
data_bin_rows = ["".join(map(lambda val:"{0:08b}".format(val,),row)) for row in data_rows]
pixel_values = zip(*data_bin_rows)
print pixel_values[0],"=",int("".join(pixel_values[0]),2) #pixel0

cant speak to its speed ... but its probably reasonable if your not doing it like a million times a second ... should be much faster than the serial read in anycase ...

回答2:

Using numpy should be fast enough or you need to write it with assembler:

import numpy
input_array = numpy.zeros((32,14,256), dtype="B")
output_array = numpy.zeros((32,8,256), dtype='int16')
for j in range(8):
    bits = (input_array[:,:,:]>>j) & 1
    for i in range(14):
        output_array[:,j,:]|= bits[:,i,:] << i

来源：https://stackoverflow.com/questions/23479288/convert-rows-of-hexadecimal-values-to-binary-vertically-by-column

标签

python

c++

optimization

numpy

bit-manipulation